
BACKGROUND OF THE INVENTION 



1. Field of the Invention. 
5 This invention relates generally to calculators, and 

more specifically, to a calculator capable of recognizing 
handwritten input . 



2. Description of Related Art. 
10 At times, nearly everyone still uses scratch paper to 

perform manual calculations. These are skills that are 

taught at an early age. Typically, a person writes the 

numbers on the paper . in a horizontal arrangement with an 

equal sign: 
15 111 + 222 = 333' 

Alternatively , the person writes numbers on paper in a 

vertical arrangement with decimals aligned and a line 

underneath: 
111 

20 +222 
333 

The person does the calculation manually using 
techniques learned early in life, and then writes the 
result (which is sometimes wrong) below the line or to the 
25 right of the equal sign. If more calculations are required 
using the result, then the person places them near the 
result to avoid writing twice. Some information may be 
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left off the paper. The person can copy numbers from one 
part of the paper to another. The paper remembers 
everything done, and displays the whole calculation . With 
sufficient training or tools, anyone can do any 
5 mathematical operation . 

Because of the drudgery involved and because of the 
real possibility of errors, many calculations are performed 
with calculators. Calculators are typically mechanical or 
electric devices. Electronic calculators are fast and 

10 accurate, and the advent of integrated circuits have 

provided more powerful calculators capable of sophisticated 
functions, including programmable calculators. Today, 
calculators can be specialized for particular applications 
including basic engineering, scientific or statistical 

15 applications. 

Unfortunately, as calculators become more powerful and 
capable, they become harder to use. Often, function keys 
on the calculator may represent two, three or more 
functions . A person has to read and understand a thick 

20 reference manual in order to use the calculator. No wonder 
people may believe you need a Ph.D just to operate the 
calculator. 

The present invention takes the concept of a 
calculator in a full circle back to its beginnings, i.e., 
25 the use of scratch paper. The present invention simulates 
a "smart pad" of scratch paper capable of performing 



calculations that a person handwrites in a natural 
notation. The present invention recognizes all types of 
calculations and mathematical functions, thereby providing 
the same capabilities as the most powerful calculators. 

5 

SUMMARY OF THE INVENTION 
To overcome the limitations in the prior art described 
above, and to overcome other limitations that will become 
apparent upon reading and understanding the present 

10 specification, the present invention discloses a pen-based 
calculator that recognizes handwritten input. The 
calculator comprises a display simulating a sheet of paper, 
and a stylus simulating a pen. The user writes a 
calculation on the calculator as if it were a piece of 

15 scratch paper. The calculator uses handwriting recognition 
to identify the various elements of the calculation, 
performs the calculation, and then displays the result at 
an appropriate location. 

20 BRIEF DESCRIPTION OF THE DRAWINGS 

Referring now to the drawings in which like reference 
numbers represent corresponding parts throughout: 

Figure 1 illustrates the packaging for a SmartPad 
calculator according to the present invention; 
25 Figure 2 illustrates the electronic components 

comprising the present invention; 
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Figure 3 illustrates the. scratch paper metaphor as 
presented to the user on the screen of the SmartPad 
calculator; ^ 

Figure 4 illustrates a "training palette" as presented 
5 to the user on the screen of the SmartPad calculator; 

Figure S^illustrates counter functions as presented to 
the user on t^e screen of the SmartPad calculator; 

Figure 6 illustrates conditional functions as 
presented to the user on the screen of the SmartPad' 
10 calculator; ^ 

Figure 7 illustrates spreadsheet functions as 
presented to the user on the screen of the SmartPad 
calculator; ^ 

Figure 8 illustrates a copy operation as presented to 
15 the user on th€ screen of the SmartPad calculator; 

Figure 9 illustrates total and subtotal calculations 
as presented to the user on the screen of the SmartPad 
calculator; ^~ 

Figure 10 illustrates calculations linked indirectly 
20 through a copy operation as presented to the user on the 



screen of the SmartPad y:alculator; 

Figures llh and 11B illustrate inter-page links 
between calculations as presented to the user on the screen 



of the SmartPad calculator; 
25 Figures 12A and 12B illustrate a selection operation 

as presented to the user on the screen of the SmartPad 





calculator; ^ ^ ^ 

Figures 13A~, 13B, and 13C illustrate how corrections 
are made on the screen of the SmartPad calculator by 
overwriting the number; ^~ ^~ ^ 

Figures l^A, 14B, 14C, 14D, and 14E illustrate how 
corrections are made on the screen of the SmartPad 
calculator by inserting digits in the number; 

Figures 15A, 15B, and 15C illustrate how corrections 
are made on the screen of the SmartPad calculator by 
10 deleting digits f^rSm y the numbers; 



.ts trom jL\ 
16A, 16B f 



Figures 16A, 16B f and 16C illustrate how corrections 
are made on the screen of the SmartPad calculator by 
interchanging digits in the~numlDers ; ^ 

Figures 17a/^17bT 17C/ 17D, and 17E illustrate how 
15 corrections are made on the screen of the SmartPad 

calculator by erasing digits from the numbers and replacing 
them with new digits j^-— £ 

Figures 18^ 18B, and 18C illustrate how corrections 
are made on the screen of the SmartPad calculator by 
20 overwriting the ^pera£or; ^~ f f ^ 

Figures ISA, 19B, 19C, 19D, 19E, and 19F illustrate 
how corrections are made on the screen of the SmartPad 
calculator by inserting operators and numbers; 

Figures 20A^ 20B, and 20C illustrate how corrections 
25 are made on the screen of the SmartPad calculator by 
deleting operators and numbers from the calculations; 
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Figures 21A, 2 IB, 21C, 2 ID, and 2 IE illustrate how 
corrections are made on the screen of the SmartPad 
calculator by replacing operators and numbers in the 
calculations ; f 

Figures 22A, 22B, and 22C illustrate how calculations 



are stopped on the screen of the SmartPad calculator by 

erasing the resuLt and/or calculating symbol; 

/ ( I 
Figures 23A, 23B, and 23C illustrate how corrections 

are made on the screen of the SmartPad calculator by 

10 interchanging operators and- numbers in calculations ; 

f ( f ' f I 
Figures -2#A, 24B, 24'C, 24D, and 24E illustrate how 

corrections are made on the screen of the SmartPad 

calculator by selecting a part thereof and then moving it 

to a new position-: s 

15 Figures 25A and 25B illustrate how links are 

established on the screen of the SmartPad calculator by 

position or by copyina;. 



r r f { / 

Figures 26A, 26B, 26C, 26D, and 26E illustrate how 



position links are broken on the screen of the SmartPad 
20 calculator; ^ 

Figures 27A, 27B, and 27C illustrate how copy links 
are broken on the screen of the SmartPad calculator by 

erasing or deleting the copy; s / 

/(if I 
Figures 28A, 28B, 28C, 28D, and 28E illustrate how 

25 copy links are broken on the screen of the SmartPad 

calculator by selecting and moving the copy out of the 





second calculation; 

Figures 2S^A, 29B, 29C, 29D, and 29E illustrate how 
position links are made on the screen of the SmartPad 
calculator; _ r ^ 



(OA, 30B, 



Figures 30A, 30B, and 30C illustrate how copy links 

are made on the screefPof the SmartPad calculator; 

S~ ( ( ( 
Figures 3>1A, 3 IB, 31C, and 3 ID illustrate how label 



corrections are made on the screen of the SmartPad 
calculator; 

10 Figures 32A, 32B, 32 f C, 32D, 32E and 32F illustrate how 

object position changes are made on the screen of the 
SmartPad calculator: 



Figures 33A, 33B, and 33C illustrate how to move 
numerous objects by just moving one result on the screen of 
15 the SmartPad ca^udlitor^ ^ 

Figures 34A, 34B, and 34C illustrate how the move of 
the original or copy does not affect the position of the 
other object on the screen of- the SmartPad calculator; 

r r r r f i 

Figures 35A, 35B, 35C, 35D, 35E, and 35F illustrate 
20 how changes to the size of objects are made on the screen 
of the SmartPad calculator; 

Figures 36A, 36B, 36C, and 36D illustrate how changes 
are made to the orientation of the screen of the SmartPad 
calculator; I 

/ r f I 

25 Figures 37A, 37B, 37C, and 37D illustrate how 

scrolling is performed on the screen of the SmartPad 



calculator; ^ 

Figure 38 illustrates a table of contents page for a 
pad as presented to the user on the screen of the SmartPad 
calculator; ^ ^ 

5 Figures 39A, 39B, 39C, and 39D illustrate how pages 

can be split on the screen of the SmartPad calculator; 

Figure 4'0 is a flow chart describing the logic of the 
Program function in the preferred embodiment of the 
SmartPad calculator; 

/ 

10 Figure 41 is a flow chart describing the logic of the 

Initialize function in the SmartPad calculator; 

Figures 42A and 42B together are a flow chart 
describing the logic of the Update-Page function in the 
SmartPad calculator; 
15 Figure 43 is a flow chart describing the logic of the 

Respond-to-Pen^f unction in the SmartPad calculator; 

Figure 44 is a flow chart describing the logic of the 
Handle-Pen-In-TAtle funeftion in the SmartPad calculator; 
Figures 45A and 45B together are a flow chart 
20 describing the logic of the Handle-Pen-In-Palette function 
in the SmartPad calculator; 

^^ FigurQ 46 i c a flow chart describing the logic of the 
Handle-Pen- Iri-Menu function in the SmartPad calculator; 
Figures 47A and 47B together are a flow chart 
25 describing the logic of the Handle-Pen- In-Page function in 
the SmartPad calculator; 
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Figure 48 is a flow chart describing the logic of the 
Handle-Written-Symbol function in the SmartPad calculator; 

Figures 49A and 49B together are a flow chart 
describing the logic of the Do-Action function in the 
SmartPad calculator; ^ s , ^jC 



Lcuiator; ^ s , t j k 



Figures 5 OA, 5 OB, -en* 5 OC together are a flow chart 
describing the logic of the Add-Object function in the 
SmartPad calculator; 

^^X ffiguro 51 io a flow chart describing the logic of the 

10 Generate-Numbe^ function in the SmartPad calculator; 

Figure 52 is a flow chart describing the logic of the 

Generate-Horizontal-Calculation function in the SmartPad 

calculator; j 

Figure 53 is a flow chart describing the logic of the 

15 Generate-Vertical-Calculation function in the SmartPad 

calculator; _ ^ . _ / 

f f, a~A 5MC 

Figures 54A*eftd 54B together are a flow chart 
describing the logic of the Generate-Copy-Calculation 
function in the SmartPad calculator; 
20 ~^^> Figurc 55 - is a flow chart describing the logic of the 
Find-Object-To-Add-To-Horizontal-Calculation function in 
the SmartPad calculator; / 

r p*~<k sec 

Figures 56Ayatt4 SSB^ogether are a flow chart 
describing the logic of the Find-Ob ject-To-Add-To-Vertical- 
25 Calculation function in the SmartPad calculator; 

r 

Figure 57 is a flow chart describing the logic of the 
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Create-Calculation-Result function in the SmartPad 
calculator; and 

■ Figure 5 8 is a flow chart describing the logic of the 
Calculate function in the SmartPad calculator. 

5 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
In the following description of the preferred 
embodiment, reference is made to the accompanying drawings 
which form a part hereof, and in which is shown by way of 
10 illustration a specific embodiment in which the invention 
may be practiced. It is to be understood that other 
embodiments may be utilized and structural changes may be 
made without departing from the scope of the present 
invention. 

15 

1 . PURPOSE 

Figure 1 illustrates the packaging for a SmartPad 
calculator according to the present invention. The 
SmartPad calculator 10 consists of a display 12 simulating 

20 a sheet of paper and a stylus 14 simulating a pen. The 
user writes a calculation on the SmartPad calculator as 
though it was a piece of scratch paper. The SmartPad 
calculator works like scratch paper, but does the 
calculations automatically rather than requiring the user 

25 to perform manual calculations. The SmartPad calculator 

combines the best features of manual calculations with the . 
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power and accuracy of electronic calculators. The SmartPad 
calculator uses handwriting recognition to determine what 
the calculation is, performs the calculation, and then - 
displays the result at an appropriate location (below the 
5 line or after the equal sign) . 

* 

2 . HARDWARE 
Figure 2 illustrates the electronic components 
comprising the present invention. The SmartPad calculator 
10 preferably comprises a flat hardware package consisting of 
a screen display 12 combined with digitizer input surface, 
a stylus 14, a processor 16 with access to read-only 18 and 
read-write memory 20, and an electrical power source 22. 
Optionally, the SmartPad calculator may have removable 
15 memory 24, a data interface 26, and a printer 28. 

Package 

The SmartPad calculator works best in a device that 
lies flat, like a notebook. This is because the user 

20 writes directly onto the surface of the screen. Typically, 
the user places the SmartPad calculator on a desk, or holds 
the SmartPad calculator in their hands or on their lap. 
The SmartPad calculator can be as small as a credit card or 
as large as a pad of accounting paper. The SmartPad 

25 calculator works in any orientation, and the orientation 
can be changed while in use. 

11 
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Screen 

The SmartPad calculator's screen takes up nearly all 
of one side of the package. In fact, to the user, the 
SmartPad calculator is not much more than a screen. 
5 Ideally, the screen makes the SmartPad calculator look like 
a sheet of paper. However, current low-power electronic 
display technology (e.g., LCDs) is not really as bright as 
paper, although this should be corrected as display 
technology advances. 

10 The SmartPad calculator's screen is preferably a 

rectangular grid of individually addressable pixels, with a 
resolution of at least 30 pixels per inch. Ideally, the 
pixels are square. Since the SmartPad calculator screen is 
combined with a digitizer input surface, it is capable of 

15 sensing which pixel is under the stylus tip when the stylus 
is in contact with the screen's surface. 

Stylus 

The user "writes" calculations on the SmartPad screen 
20 with a stylus. The stylus feels like a pen or pencil, but 
dispenses "electronic ink" and not real ink. The user 
writes directly onto the screen, and the electronic ink is 
displayed directly underneath the tip of the pen as the 
user writes . 

25 The screen surface makes stylus writing feel like 

writing with a real pen on real paper, so the user writes 

> 
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naturally. Preferably the SmartPad calculator has a dock 
to hold the stylus when not in use. Additionally, in the 
preferred embodiment, power to the screen is turned on 
automatically when the stylus is removed from the dock, and 
5 then turned off automatically when the stylus is returned 
to the dock. 

1 Processor 

The SmartPad calculator has a general-purpose 
10 microprocessor to run the software that provides most of 
the SmartPad calculator's functionality. Low-power 
versions of the INTEL 80x86, MOTOROLA 680x0, as well as 
other processors, are suitable for the SmartPad calculator. 
Further, special-purpose circuits or microprocessors (e.g., 
* 15 neural networks, ASICs, etc.) may also be used. 



Read-Only Memory 

The SmartPad calculator preferably uses a read-only 
memory (ROM) to hold the software. The ROM may also hold 
20 fonts (character images) for displaying, and default 
character descriptions for a handwriting recognizer. 

Read-Write Memory 

The SmartPad calculator preferably uses a low-power 
25 read-write memory (RAM) to hold information which changes. 
This may include the screen image, calculations, and 
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trained character descriptions for the recognizer. Some or 
all of the RAM may remain powered even when the SmartPad 
calculator is turned off, so the pad and training are not 
lost. 

5 

Removable Memory 

The SmartPad calculator may provide removable (off- 
line) memory. Removable memory holds copies of the user's 
pads and training. Removable memory lets the user have an 

10 unlimited number of pads. It also allows pads to be backed 
up for safety or archived for later retrieval. Finally, 
removable memory provides a way to transfer pads and 
training from one SmartPad calculator to another. 

Current technology offers two kinds of removable 

15 memory that are most suitable for the SmartPad calculator: 
magnetic cards and floppy disks. Compared to floppy disks, 
magnetic cards are smaller, lighter, and use less power. 
However, floppy disks can be read by personal computers. 

20 Data Interface 

The SmartPad calculator may provide a data interface. 
The data interface allows information to be transmitted to 
and received from other devices supporting the same 
interface. The other device might be another SmartPad 

25 calculator, a computer, a printer, or a fax machine. By 
transferring information to another SmartPad calculator, 

14 



the user can share pads with another user, or move pads to 
larger or more portable hardware. By transferring 
information to a computer, the user can use a personal 
computer version of the SmartPad calculator, which may 
5 provide keyboard input of numbers, faster calculations, 
more printing options, and exporting to spreadsheets. By 
transferring information to a printer or fax machine, 
calculations can be printed or sent to another person. 

The SmartPad calculator's data interface might be an 

10 infrared transceiver, a serial (RS-232) cable, or a direct 
telephone connection. An infrared transceiver uses little 
power and takes little space, but typically is only 
compatible with other SmartPad calculator devices in the 
immediate vicinity. A serial interface uses more power, 

15 takes more space, and is not well standardized. However, a 
serial interface connects to computers, printers, and 
modems. Finally, a direct telephone connection may connect 
v to fax machines and, through modems, to computers. 

20 Printer 

The SmartPad calculator may provide a built-in printer 
to copy the screen or other data onto paper. Printing 
gives the user a permanent record of a calculation, 
suitable for backup, archiving, or sending to another 
25 person. Although plain paper 'printing is preferable, plain 
paper printers are bigger, more expensive, and use much 
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power. Thermal printing onto a special waxed paper does 
not have these problems, but the paper is harder to get, is 
easily marked, and fades. Alternatively, the user can 
print through the data interface, as described above. 

5 

Power 

Of course, the SmartPad calculator requires electrical 
power. Typically, this is either AC (plugging into a 
standard electrical outlet) or DC (installing a battery 

10 inside the package) or both. The battery may be either a 
common watch or calculator battery, a standard radio 
battery, or a rechargeable NiCad battery. With an AC 
supply, batteries are still useful to keep RAM powered 
while the SmartPad calculator is not plugged in, or to 

15 provide portability. Unless the batteries are 

rechargeable, they should keep the SmartPad calculator 
powered a long time, to minimize the frequency of battery 
changes. The SmartPad calculator also has an internal 
power "reservoir" (capacitor) which keeps RAM powered while 

20 the battery is being changed or charged. 



4 . PLATFORMS 
The SmartPad calculator runs on dedicated devices, 
pen-based computers, and even on regular computers. 

25 
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Dedicated Devices 

Unlike most software applications, the SmartPad 
calculator is quite useful running on a dedicated device, 
i.e., one that only runs the SmartPad calculator software. 
5 The. smallest version of the SmartPad calculator may be the 
size of a credit card, and have a 64 x 128 pixel display, a 
small stylus, 64K ROM, and 4K RAM. It most likely has no 
removable memory, data interface, or printer. It uses 
watch batteries, which last at least a year. The pad may 
10 only have one small page. On the other hand, by reducing 
the number of components, the cost can be kept low, for 
example, $25. 

An intermediate version of the SmartPad calculator may 
be a bit narrower than a letter envelope, and have a 256 x 

15 512 pixel display, a normal stylus, 128K ROM, and 64K RAM. 
Such a version may provide some way to print calculations, 
e.g., a thermal printer with an infrared transceiver 
interface. It may also use radio batteries, which last a 
few months . The cost of a such a unit could be kept as low 

20 as $150. 

The largest version of the SmartPad calculator may be 
the size of a sheet of paper (maybe larger), and have a 768 
x 512 pixel display, 256K ROM, and 128K RAM. It may 
provide some kind of computer interface and have an option 
25 for a built-in printer. It may also use AC power and have 
rechargeable batteries for portability. It could include 
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other features, such as note-taking, time-management, and 
phone directory. The cost of a such a unit could be kept 
as low as $500. 

5 Pen-Based Computers 

The SmartPad calculator is a useful application on 
pen-based computers. Pen-based computers are well known in 
the art, including such examples as Tandy Corporation's 
Grid unit, NCR Corporation's System 3125, and Apple 
10 Computer's Newton Personal Digital Assistant (PDA). 

Pen-based computers use a stylus as their primary 
input device. Pen-based computers can be as small as a 
paperback novel or as large as a pad of letter-size paper. 
They are useful for filling out forms, finding and viewing 
15 information stored electronically, and taking notes. 

Typically, pen-based computers operate under the 
control of a specialized operating system, for example, 
Microsoft Corporation's Pen Windows™, GO Corporation's 
PenPoint™, and Communications Intelligence Corporation's 
20 PenDOS™. The means for recognizing handwritten input is 

embedded in the operating system, and as such, applications 
need little or no modification to run under such operating 
systems. Mouse and keyboard operations are usually 
emulated by the movement of or tapping of the stylus on the 
25 writing surface of the computer. 

With the SmartPad calculator, pen-based computers are 
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also useful for simple- to-moderately complex calculations 
that otherwise require a calculator or spreadsheet. Pen- 
based computers and their operating systems might "bundle" 
a minimal version of the SmartPad calculator. Users could 
5 buy a more sophisticated version separately. On a pen- 
based computer, the SmartPad calculator uses the computer's 
available hardware, including removable memory, data 
interfaces , and printers . 

10 Regular Computers 

The SmartPad calculator may also be useful on regular 
computers, especially those with graphical user interfaces. 
On a regular computer, the data from a SmartPad calculator 
is easily printed, backed up, archived, exported to 

15 spreadsheets, or sent to other computers or SmartPad 

calculator devices . The SmartPad calculator on a regular 
computer is also an attractive alternative to spreadsheets 
for users. The disadvantage of the SmartPad calculator on 
a regular computer is the lack of handwriting recognition 

20 software and hardware. This requires a user interface that 
is significantly different, using the keyboard for number 
and operator entry and the mouse or cursor keys for 
placement, selection, and command selection. 

25 



19 




5. USER INTERFACE 
The SmartPad calculator's user interface is 
characterized by "scratch paper" and "notepad" metaphors, 
along with real-time handwriting and calculation 
5 recognition. In addition, the user trains the SmartPad 
calculator with a "learning palette". 

Scratch Paper Metaphor 

Figure 3 illustrates the scratch paper metaphor as 

10 presented to the user on the screen 12 of the SmartPad 

calculator. When a new page is created, it is completely 
blank. The user writes out calculations in a natural 
manner, as if they were being done manually. Moreover, 
calculations can be written horizontally or vertically. 

15 Unlike real scratch paper, the SmartPad calculator 

computes the result, i.e., 6.0 and 24.98, automatically as 
soon as an equals sign (" = ") is drawn to the right of the 
numbers. For vertical calculations, the SmartPad 
calculator computes the result, i.e., 30.98 and 2.17, as 

20 soon as a "result line" is drawn underneath the numbers. 
As with real scratch paper, the user can write 
calculations anywhere on the screen, and in any 
orientation. In addition, numbers and symbols can be 
written in any reasonable size. Unlike scratch paper, the 

25 SmartPad calculator might allow screens to be of arbitrary 
size, so the user never runs out of space to add more 
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calculations (until out of RAM or virtual memory) . 

Notepad Metaphor 

The SmartPad calculator presents a notepad metaphor on 
5 the screen 12 , which means that it operates like a pad of 
paper. Each piece of paper is one page of the pad. The 
user can create a new blank page or throw away an old used 
one. The user can have more than one page in the pad 
filled with calculations, but the screen only shows one 

10 page at a time. Unlike real pads, the SmartPad calculator 
allows an arbitrary number of pages (until out of RAM) . To 
identify pages, each page is numbered, i.e., "4" in Figure 
3, and the user may write in a title, i.e., "order for John 
Doe" in Figure 3. The user can switch pages at any time; 

15 when switching, the user can go from one page to the next 
in sequence, or choose a page from a table of contents 
showing all the titles and page numbers . 

Real-Time Recognition 

20 The SmartPad calculator uses handwriting recognition 

technology to identify the digits and symbols written by 
the user. The SmartPad calculator recognizes characters in 
real time (as soon as the user writes them) . Once 
recognized, the "electronic ink" is replaced on the screen 

25 by a well-formed character of approximately the same size; 
this gives the user immediate feedback on the accuracy of 
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the recognition. 

In the SmartPad calculator , character recognition is 
fast enough to keep up with a user writing as quickly as 
humanly possible* Character recognition is also a 
5 background process, so the writing of the next character 
never has to wait until the previous character is 
recognized. 

After character recognition has occurred and the 
character identified as a digit, the SmartPad calculator 

10 uses a "number recognizer" to identify which number the 
digit belongs to. The number recognizer looks for any 
other numbers that are close to the new digit; if one is 
found, the SmartPad calculator incorporates the digit into 
the number. If no other number is close, the SmartPad 

15 calculator creates a new single-digit number. Once 

recognized as part of a number, the SmartPad calculator 
makes all characters the same size, aligns them 
horizontally, and adjusts the spacing between them. These 
adjustments show the user that the number was properly 

20 recognized, and helps keep the user's writing on track. 
As soon as the SmartPad calculator recognizes a 
calculating symbol (such as the equal sign or the result 
line), the SmartPad calculator uses a "calculation 
recognizer" to identify the numbers and operators that are 

25 part of the calculation. Once a calculation is recognized, 
all of its characters are made uniform in size, aligned 
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horizontally and/or vertically , and spaced evenly . 
Anything nearby but not a part of the calculation is pushed 
away. This gives the user immediate feedback on the 
accuracy of the calculation recognizer. Thereafter, the 
5 SmartPad calculator does the calculation and displays the 
result in the appropriate place. 

Training Palette 

Figure 4 illustrates a "training palette" 30 as 

10 t presented to the user on the screen 12 of the SmartPad 
calculator. This palette is displayed at the top of the 
screen 12, and shows each of the characters and gestures 
which the SmartPad calculator recognizes. If the SmartPad 
calculator incorrectly recognizes any character or gesture, 

15 the user may press the stylus on the correct one in the., 
palette 30, and the SmartPad calculator makes the 
appropriate change to the page. The SmartPad calculator 
also remembers how the user wrote that character, so next 
time it is recognized correctly. This "trains" the 

20 SmartPad calculator to recognize the user's handwriting, no 
matter how bad it is . To minimize the recognition error 
rate, users should spend a little time training a new 
SmartPad calculator device. Once trained, it is faster to 
make corrections by rewriting the character better. 

25 
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6 . NUMBERS 

Numbers are obviously an important part of the 
SmartPad calculator. Numbers consist primarily of digits. 
However, there is more to numbers than just digits, and 
5 there are different kinds of numbers. 

Number Entry 

The user normally enters a number into the SmartPad 
calculator by writing its digits in sequence from left to 

10 right on the screen. The SmartPad calculator's number 

recognizer does not require this particular order, but does 
require that digits be placed on the page in reasonable 
proximity to each other, mostly horizontal, with higher- 
order digits to the left of lower-order digits. The digits 

15 can be any size (within reason), but all the digits of a 
number should be roughly the same size. A simpler 
alternative number recognizer would just add digits to the 
number in the order written, but then the user could not 
enter digits out of order. The SmartPad calculator may 

20 limit the number of digits in a number. If there is a 
limit, it is not less than nine digits in the smallest 
version of the SmartPad calculator, and is higher in other 
versions . 

Besides digits, the user can write decimal points and 
25 thousand separators (periods and commas in the U.S.). The 
SmartPad calculator recognizes and displays well-formed 
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versions of these characters as they are written. The user 
is not required to enter a decimal point unless the number 
contains non-zero fraction digits; the SmartPad calculator 
adds the decimal point and zero fraction digits if 
5 appropriate. The user is also not required to enter 

thousand separators; the SmartPad calculator adds them to 
the number when it is fully recognized (after the last 
digit is entered) . 

10 Number Alignment . 

Numbers in the SmartPad calculator are subject to 
baseline, size, and spacing alignments. Alignment occurs 
as the number is recognized. Baseline alignment means that 
the bottoms of the digits are aligned to a single imaginary 

15 (e.g., invisible) horizontal line. Size alignment means 
that all the digits of the number have the same size. In 
other words, the widths and heights of the digits are 
, identical. Finally, spacing alignment means that the 
digits are spaced uniformly. Numbers must also be aligned 

20 to other parts of a calculation, but the SmartPad 

calculator does this when the calculation is recognized. 

Zero Fill 

To simplify number entry, the SmartPad calculator 

25 provides an automatic zero-fill function in certain 

situations; sometimes, the user may have to invoke the 
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zero-fill function by writing a dash or other special 
characters. Zero-fill means that one or more zeroes are 
appended to a number without the user writing them. When 
writing a horizontal calculation (or the first number of a 
5 vertical calculation) , the zero-fill function is less 

automatic. If the number includes a thousand's separator , 
the zero-fill function adds zeroes until there are three 
digits after the separator. For example, writing H 3,2- M 
results in "3,200". After a decimal point, the zero-fill 

10 function adds enough zeroes to match the number of fraction 
digits in the last number entered; the default is two 
fraction digits . For example , writing " 4 . - " could result 
in "4.00". In other situations, the zero-fill function has 
less meaning, and results in adding a single zero. For 

15 example, writing "5-" would result in "50". 

In vertical arrangements of numbers, the zero-fill 
function is more automatic. In that case, the vertical 
placement of the digits determines how many zeroes are 
added. The SmartPad calculator keeps unit's digits aligned 

20 vertically, and keeps the same number of fraction digits in 
each number. Thus, if the user first writes 
"2,845,605.15", and then below that writes "79-", with the 
"79" underneath the "84" in the first number, the SmartPad 
calculator produces "790,000.00". 

25 After filling out a number with zeroes, the zero 

digits are like any other digit written by the user; 
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therefore, the user can change any zero by writing over it. 

Whole Numbers 

Whole numbers are integers that are not negative. 
5 Whole numbers include "0", "1", "52 M , and - 3 , 195 , 384 " . The 
user enters whole numbers by simply writing the digits, 
optionally including thousand separators. Whole numbers 
are displayed as written, except that thousand separators 
are automatically included after the number is completed. 

10 

Real Numbers 

Real numbers are numbers that include a fraction part. 
Real numbers include "6.4", "7,498. 21", "0.000", and "5.". 
The user enters real numbers by writing the digits and the 
15 decimal point, optionally including thousand separators. 

Real numbers are displayed as written, except that thousand 
separators are automatically included. 

Negative Numbers 

20 The user does not enter negative numbers, but instead 

writes symbols that cause a number to be negated in a 
calculation. There are two ways to negate a number: (1) 
either write a minus sign in front of it, or (2) enclose it 
in parentheses. The SmartPad calculator displays the minus 

25 sign or parentheses as the user writes them. When the 

result of a calculation is a negative number, the SmartPad 
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calculator displays it in the same way the user has negated 
numbers elsewhere; the default is to use the minus sign. 

Currencies 

5 Numbers representing monetary amounts (currencies) are 

the most common type of number appearing in everyday 
calculations. In the U.S., currencies are normally 
prefaced with a dollar sign H $' f and have two fraction 
digits (e.g., $211.65). The SmartPad calculator is capable 

10 of recognizing dollar signs and carrying them through 

calculations, automatically prefixing dollar signs to the 
results where appropriate. Versions of the SmartPad 
calculator for other countries would recognize the 
appropriate currency symbol and display the results in the 

15 right format. 

Fractions 

Users might sometimes want to enter fractions instead 
of the decimal equivalent. Often, fractions are more 
20 accurate. The SmartPad calculator supports fractions, and 
allows the user to write them in a natural manner. 

Calculations with fractions retain full accuracy, so 
the results may also contain fractions . The SmartPad 
calculator does not convert fractions to decimal, as this 
25 may produce round-off error. Fraction results are reduced 
(e.g., "2/4" becomes "1/2"). 
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Other Numbers 

There are many other kinds of numbers that users work 
with and that are supported by the SmartPad calculator. 
These include measurements (such as feet and inches, for 
5 example: 9* 10"), dates (such as 5/18/92), times (such as 
11:31 AM), scientific notation (such as 4.123xl0 8 ), complex 
numbers (such as 12+5i), numbers in non-decimal bases (such 
as 274 8 or 4E7A 16 ), and even matrices. These numbers 
require special algorithms for performing arithmetic 
10 operations. Some kinds of numbers would only be supported 
in specialized versions of the SmartPad calculator (for 
example, an engineer's version). 

Counters 

15 Figure 5 illustrates counter functions 32-38 as 

presented to the user on the screen 12 of the SmartPad 
calculator. Sometimes the user requires item counts as 
input to a calculation, for example, adding up the value of 
some change by counting the numbers of pennies, nickels, 

20 dimes, and quarters. Typically, the user sorts the coins 
by type, and then counts each type separately. The 
SmartPad "counters" 32-38 are icons that keep track of how 
many times they are tapped by the stylus. Counters 32-38 
start at zero, so that each time the stylus touches the 

25 screen 12 of a counter 32-38, its value is increased by 

one. There is also a way to decrease its value by one, in 



case the tap was a mistake. Ideally, each change in value 
would be accompanied by audio feedback. The value of the 
counter 32-38 is used in calculation like any other number. 
With counters 32-38, the user does not have to sort the 
5 coins. Instead, a separate counter 32-38 is created for 
each type of coin, and the coins are then processed in any 
order; wherein the appropriate counter 32-38 is tapped once 
for each coin. While this is going on, the SmartPad 
calculator can be computing a running total. 

10 

7 . OPERATORS 

Operators are the parts of calculations that are not 
numbers. Operators specify the operations to perform on 
the numbers. Most operators are single symbols, such as 
15 the plus sign "+". As with digits, the SmartPad calculator 
uses handwriting recognition technology to determine which 
operator is written on the screen 12 . 

Arithmetic Operators 

20 The SmartPad calculator supports the four basic 

arithmetic operators: add, subtract, multiply, and divide. 
Addition is represented by the plus sign "+ M , and 
subtraction by the minus sign Multiplication is 

represented by the letter "x" or a raised period • . 

25 Multiplication is also represented by the at sign , as 
in "5 @ $1.05 = $5.25". Division is represented by a slash 



"/" or a divide sign "t". In addition, the SmartPad 
calculator supports percentages, represented by the percent 
sign "%". Percentages are usually multiplied, added, or 
subtracted, as in the following: "3.00 x 6% = 0.18", "3.00 
5 + 6% = 3.18" , and M 3.00 - 6% = 2.82". Numbers may be 

divided by percentages as well, as in "3.00 t 6% = 50.00", 
which tells you that 3 is 6% of 50. 

All but the smallest version of the SmartPad 
calculator supports the square root operation. However, 
10 the natural representation of square root is not a simple 
character; instead, the user writes: 



|^3^°^ v/ 30 + 19 



A line is drawn over the entire expression whose value 
is input to the square root operation. A simplified 
15' version of the SmartPad calculator might use parentheses 

instead of the line, as in "v®(30 + 19) " , but this is less 

natural . 

The SmartPad also supports the power operation (e.g., 
squaring). Unlike other operations, there is no symbol for 

20 the power operation; instead, it is represented by making 
the power a superscript, as in "15 2 ". To support this 
representation, the number recognizer must be careful not 
to consider the exponent part of the base. The calculation 
recognizer then can identify the power operation based on 

25 the relative placement of the two numbers. 
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Ordering Operators 

Ordering operators control the order in which 
operations are performed. Usually, operations that are 
performed first are enclosed in parentheses, as in: 
5 5 x (3 + 4) = 35 

Besides parentheses, the SmartPad calculator also 
supports brackets " [ ] " and braces "{}". Ordering operators 
only apply to horizontal calculations; vertical 
calculations are always performed from top to bottom. 

10 

Special Functions 

Specialized versions of the SmartPad calculator may 
support a number of mathematical functions. Business users 
could use time-value-of -money functions, which compute 

15 present and future value, loan payments, etc. Accountants 
could use depreciation and amortization functions. 
Engineers could use trigonometric and logarithmic 
functions . Some functions have graphic representations 
(e.g., integration), but most are represented by letters 

20 and parentheses, as in: 
sin(25°) 

To handle functions like this, the handwriting 
recognizer must recognize letters as well as digits and 
operators . An alternative embodiment would recognize the 
25 entire function name (e.g., "sin"). This may be more 
accurate, and will allow names to be written in script 



instead of individually printed letters. 
Conditionals 

The SmartPad calculator keeps track of the logical 
5 structure of the calculations on a page. This allows the 
user to go back and fix any mistakes without redoing the 
entire calculation. This also means the SmartPad 
calculator page is a template in which the user can enter 
input values and view output results. To be useful, this 
10 feature requires conditional functions , which are functions 
that use different formulas depending on one or more 
conditions. For example , computing the absolute value 
requires a conditional function of the form: 
if xxx < 0 then -xxx else xxx 
15 where the second and third "xxx" are copies of the first 

"xxx." To represent conditional functions, those with math 
training use braces to enclose a vertically arranged list 
of conditions as illustrated in Figure 6 . 

However, recognizing the layout in Figure 6 may be 
20 challenging. Alternatively, the SmartPad calculator may 
use the spreadsheet representation (an "if" function) as 
illustrated in Figure 7. 

Another common conditional function is table lookup. 
The SmartPad calculator supports various kinds of table 
25 lookup. The simplest is to select a single value from a 
list of values, given the index of the value in the list. 
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A more complex lookup interpolates on a table of function 
arguments and results. The table lookup functions are 
represented by the name of the table, as in: 
tax ($20,000) 

5 To name a table, the user creates the table and writes the ( 
name on top. 

Constants 

A version of the SmartPad calculator supporting 
10 trigonometric and logarithmic functions is not complete 
without the mathematical constants "n" and "e". Besides, 
these constants, the SmartPad calculator supports more 
obscure scientific constants, as well as metric 
conversions. 

15 

Other Operators 

In addition to the operators listed above, the 
SmartPad calculator may support numerical computation 
operations, including but not limited to, arbitrary- 

20 precision arithmetic, complex numbers, combinatorial 
functions, integer functions, matrix operations, root 
finding, function fitting, Fourier transforms, numerical 
integration, numerical differentiation, function 
minimization, and linear programming. The SmartPad 

25 calculator may also recognize symbolic computation 

operators, including but not limited to, equation solving, 
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symbolic integration, symbolic dif f erentiation, power 
series , limits, algebraic operations, polynomial expansion, 
factorization, simplification, matrix operations, tensor 
operations, list operations, and string operations. 

5 

8 . CALCULATIONS 
The heart of the SmartPad calculator's functionality 
is the calculation. There are actually different types of 
calculations, each of which works differently. The main 
10 types are "horizontal", "vertical", and "copy." 

Horizontal Calculations 

In horizontal calculations, the numbers and operators 
are arranged horizontally: 

15 25 x (3,809 + 265) - 14,266 = 87,584 

Horizontal calculations are identified by the equals 
sign "=" . As soon as the user writes an equals sign, the 
SmartPad calculator invokes the horizontal calculation 
recognizer to identify which numbers and symbols are part 

20 of the calculation, and their order within it. To identify 
components of the calculation, the recognizer looks for 
objects that are not part of another calculation, are 
arranged mostly horizontal, are reasonably close to each 
other, and include the equal sign on the right; results of 

25 other calculations are included if they have not already 
been made part of a separate calculation. The ordering of 
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the objects within the calculation is determined by their 
horizontal placement, with the leftmost object being first. 

A simpler alternative recognizer just uses the objects 
written since the last calculation, in the order written, 
5 but this does not work if the user writes out of sequence. 
This type of recognizer is appropriate only for a minimal 
version of the SmartPad calculator in which the logic of 
the calculation is not saved. 

Once a horizontal calculation is recognized, the 

10 SmartPad calculator aligns the baselines, sizes, and 
spacing of the objects in the calculation. Then, the 
SmartPad calculator does the calculation and displays the 
result. When evaluating a horizontal calculation, the 
usual rules of operator precedence are applied, except that 

15 anything within parentheses is done first. Most operators 
are infix operators, meaning that the operator is between 
the numbers it operates on. The result of the calculation 
is a number that is displayed like other numbers; it is 
aligned with the rest of the calculation but is to the 

20 right of the equal sign. If necessary, the SmartPad 

calculator moves calculations around to make room for the 
result or to maintain other alignments. 

In some cases, horizontal calculations are too long to 
fit across the page (or screen). On real scratch paper, 

25 these calculations are "broken" into two or more lines: 
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25 x (3,809 + 265) 
- 14,266 = 87,584 

The SmartPad calculator's horizontal calculation 
recognizer is capable of identifying calculations like 
5 this. 



Vertical Calculations 

Vertical calculations are calculations in which the 
^ numbers and operators are arranged vertically: 
fK 10 3,809. 

+ 265 
x 25 
- 14,266 
87,584 

15 Vertical calculations are identified by the result 

line, which is the line underneath the numbers. As soon as 
the user draws the result line, the SmartPad calculator 
invokes a vertical calculation recognizer to identify which 
numbers and symbols are part of the calculation, and their 
20 order within it. To identify components of the 

calculation, the vertical calculation recognizer first 
looks for objects immediately above the result line that 
are not part of another calculation, are mostly horizontal, 
and are reasonably close to each other. These objects are 
25 considered part of the bottom line of the vertical 
calculation. After that, the next to bottom line is 



37 




identified by looking for objects immediately above the 
bottom line meeting the same conditions. This process 
repeats until the top line is found, which means there is 
no identifiable line above it. The ordering of the lines 
5 is determined by their vertical placement, with the top 
line being first. The ordering of the objects within each 
line is determined by their horizontal placement, with the 
leftmost being first. As with horizontal calculations, a 
simpler alternative recognizer could just use the objects 

10 written since the last calculation, in the order written. 
Once a vertical calculation is recognized, the 
SmartPad calculator aligns the sizes and spacing of the 
objects in the calculation. In addition, the SmartPad 
calculator aligns the baselines of the objects within each 

15 line, as well as the vertical spacing between the baselines 
of each line. Finally, the SmartPad calculator aligns the 
right sides of the unit's digit of the rightmost number in 
each line. Then, the SmartPad calculator does the 
calculation and displays the result i When evaluating a 

20 vertical calculation, operations are performed from top to 
bottom; there is no way to change this order, even with 
parentheses. Each line of the vertical calculation is 
supposed to contain an operator followed by a number. The 
operation is applied to the result of the preceding lines 

25 and the number on the same line. The operator may be left 
out, and is assumed to be a plus sign in that case. The 




result of the calculation is a number that is displayed 
like other numbers; it is aligned with the rest of the 
calculation but is below the result line. Once the result 
is aligned, the ink of the result line is replaced with a 
5 well-formed horizontal line segment, aligned on the left 
and right with the extreme lefts and rights of the objects 
in the vertical calculation. If necessary, the SmartPad 
calculator moves other calculations around to make room for 
the result or to maintain other alignments. 



the user on the screen 12 of the SmartPad calculator. A 
copy operation is simply a way to make a copy of a number 
15 without writing it twice; the copy is placed at a different 
location on the^ page, where it can be part of another 
calculation. 

Copy operations are identified by an arrow that points 
from the original number to the copy. To create a copy 

20 operation, the user draws the arrow, starting someplace 
near the original number and ending near the center of 
where the copied number is to be located. As soon as the 
arrow is drawn, the SmartPad calculator invokes the copy 
operation recognizer to identify the original number. The 

25 recognizer looks for a number that is underneath the source 
of the arrow, or close by and in the direction opposite to 
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Copy Operations 



Figure 8 illustrates a copy operation as presented to 
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the arrow's head. Any number can be the original, even if 
it is part of another calculation (input or result). In 
particular, the number may be part of another copy 
operation, so an unlimited number of copies of the original 
5 can be made. 

Once the original number is identified, the SmartPad 
calculator makes a copy and places it underneath the head 
of the arrow. Then, the ink of the arrow is replaced with 
a well-formed arrow starting near the edge of the original 
10 number and ending near the edge of the copy. The SmartPad 
calculator may try to find a route for the arrow that 
avoids overwriting other objects on the page. 

Total /Subtotal Calculations 

15 Figure 9 illustrates total and subtotal calculations 

as presented to the user on the screen 12 of the SmartPad 
calculator. Accountants often use calculations that 
involve subtotals and a total of the subtotals. 

Total/subtotal calculations in the SmartPad calculator 

20 use regular result lines for the subtotals and then place 
another result line, resulting in a double result line, 
underneath the grand total. Since regular result lines are 
used for vertical calculations, small changes must be made 
to the way the vertical calculation recognizer works. 

25 Specifically, when there is only a single available number 
above the result line, no vertical calculation is 
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generated. Also, vertical calculations do not extend above 
other result lines, even if they are not part of another 
vertical calculation. 

Having made these changes to the vertical calculation 
5 recognizer, the SmartPad calculator can invoke the 

total/subtotal calculation recognizer as soon as the user 
writes a double result line. This recognizer searches 
above the double result line for any numbers that stand 
alone between two regular result lines. All such numbers 

10 are either subtotals of other numbers, or separate numbers 
that are not part of one or the subtotals. Once these 
numbers are identified, they are incorporated (along with 
any associated operators) into the grand total calculation. 
The grand total calculation is evaluated like a vertical 

15 calculation, and the result is placed above the double 
result line. As with vertical calculations, the numbers 
are aligned and the ink of the result lines is replaced by 
well-formed lines that are also aligned to the calculation. 



they are being evaluated. Preferably, calculations are 
animated by flashing each of its numbers and operators in 
sequence (including the result). For example, assume the 
25 user has entered the calculation of: 
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Animation 



The SmartPad calculator "animates" calculations as 



25 x ( 94 - 57 ) = 925 



The SmartPad calculator first flashes the "25", then the 
"x", the "(", the "94", the "-", the "57", the ") M , the 
" = ", and finally the result "925". The animation is slow 
enough for the user to follow, but fast enough to finish a 
5 page in a few seconds or less. By animating calculations, 
the user knows when a calculation is being performed, and 
what numbers are involved. This feedback helps the user 
make sure the SmartPad calculator recognizes the right 
calculations at the right time. 

10 

Errors 

Sometimes when doing a calculation, the SmartPad 
calculator finds a mistake or an incomplete formula, for 
example : 

15 32 x 7 - 

The kinds' of mistakes the SmartPad calculator can detect 
include: unrecognizable character, number overflow (too 
many digits), division by zero, negative square root, bad 
number format (e.g., "34.56.2"), and improper syntax (such 

20 as unmatched parentheses, missing operator, or missing 

number) . When there is a mistake, the result is undefined, 
so a few question marks " ???" are displayed on the screen 
12 instead. If the result is itself part of another 
calculation, the result of that calculation is also 

25 undefined. However, subsequent results are displayed as 
dots "...", so the user can easily locate the calculation 
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with the problem. 

The SmartPad calculator might provide informative 
diagnostics when an error occurs. These diagnostics 
indicate the kind of error, and where it is in the 
5 calculation. With this information, the user is able to 
pinpoint and correct the error quickly. 

9 . LINKS 

The SmartPad calculator can link calculations, which 
10 means that the result of one calculation is input to 

another calculation. Once a link is set up, the SmartPad 
calculator handles everything so the user does not have to 
copy results or repeat calculations. The SmartPad 
calculator saves the logical relationships, so when the 
15 user fixes a mistake or changes an input number, all 

affected results are recomputed automatically. Thus, the 
SmartPad calculator has the same capability that makes 
spreadsheets so popular. There is no limit on the number 
of links, but dependency cycles are not permitted. 

20 

Inter-Calculation Links 

Inter-calculation links are links between two 
calculations on the same page. These links are generated 
automatically by the calculation recognizer, based on 
25 placement. For example, two horizontal calculations may be 
linked: 
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3,809 + 278 = 4,087 x 3 = 12,261 

When the second equal sign is written, the horizontal 
calculation recognizer looks for numbers and operators to 
the left of the equal sign that are not already input to 
5 another calculation. The recognizer thus incorporates the 
result of the first calculation into the second 
calculation. Any calculation can link with any other 
calculation. For example, two vertical calculations may be 
linked: 
10 3,809 
+ 278 
4,087 
x 3 
12,261 

15 Another example, is a vertical calculation linked to 

two horizontal calculations: 
5 x 1.25 = 6.25 
2 x 12.00 = 24.00 
30.25 

20 Even if two calculations are not near each other, they 

can be linked indirectly through a copy operation as 
illustrated in Figure 10. 

Inter-Paae Links 
25 The SmartPad calculator . supports links between 

calculations on different pages of the same pad. This 
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allows sophisticated users to develop very complex 
calculations. An inter-page link is similar to a copy 
calculation in that a number on one page is copied onto 
another page. Inter-page links are displayed as squares 
5 containing identifying numbers, as illustrated in Figures 
11A and 11B. 

The user creates an inter-page link by drawing the 
square and writing the identifying number inside it. To 
get an unused number, a dash is drawn inside the square, 

10 and the SmartPad calculator replaces it with the next 

available identifying number. To send a number to another 
page, the user creates a copy operation (i.e., draws an 
arrow) from the number to the link object. To get a number 
from another page, the user creates a copy operation from 

15 the link object. Any number of arrows can go out of a 
link, but only one is allowed into the link. 

Inter-Application Links 

On multi-tasking computers, the SmartPad calculator 

20 supports inter-application links, which are links to values 
in other applications. For example, in a form processing 
application, the user may need to calculate the value 
entered into a field of the form. The SmartPad calculator 
can do this calculation automatically if a link is set up 

25 from the SmartPad calculator result to the form field. If 
the calculation depends on Values in other form fields, the 
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user can also set up a link from the form to the SmartPad 
calculator. Inter-application links allow the user to add 
calculations to any application. The method for creating 
and using inter-application links depends on the operating 



If a calculation must be retained for any time, the 
purpose of the calculation and the meanings of the numbers 

10 should be documented. The SmartPad calculator supports 

documentation with "labels." A label is simply ink left on 
the page. A label has no effect on the results of the 
calculations. There is no limit on the number of labels on 
a page (until out of RAM). 

15 The user creates a label by writing it. There is no 

label recognizer. Instead, anything that is not 
recognizable, or drawn over an existing label, is 
considered a label. This works well for cursive writing, 
but mistakes occur with individually printed letters (for 

20 example, the letter "I" is recognized as the digit "1"). 
To restore the original ink of a recognized character, the 
user taps the "label" command in the title area (discussed 
in more detail below in the section entitled "Pages"). To 
minimize these mistakes, the SmartPad calculator avoids 

25 recognizing characters in the midst of many unrecognizable 
characters . 



5 system. 
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11 . SELECTIONS 
Most of the time, a user of the SmartPad calculator 
simply writes or draws things and the SmartPad calculator 
recognizes them. However, a few actions (such as moving 
5 objects) require that objects be selected first. Once a 
selection is made, the action is performed with the stylus. 
Instead of dispensing ink, a "cursor" appears on the screen 
12 beneath the tip of the stylus, and the cursor acts as if 
it is attached to the stylus. Any number of objects can be 

10 selected; initially, none are selected. Objects are 

deselected when the user goes back to writing or drawing. 

To make a selection, the user simply draws a "lasso" 
around the objects to be selected, as illustrated in Figure 
12A. The lasso is either a large closed figure, or a small 

15 figure resembling a zero except for the "handle" of the 

lasso. The SmartPad calculator selects an object if nearly 
all of its interior is inside the lasso. Selections are 
cumulative, so more objects are added to the selection by 
lassoing them. To deselect objects, the user lassoes them. 

20 As long as no unselected objects are inside the lasso, the 
objects in the lasso are, removed from the selection. To 
deselect everything, the user taps the stylus anywhere. 

The SmartPad calculator flashes the selected objects 
so the user can see which ones they are. In addition, the 

25 SmartPad calculator displays four handles (square dots) in 
a rectangular arrangement around the selected objects, as 
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illustrated in Figure 12B. The handles are used to change 
the sizes of the selected objects. 



mistakes and fix them. Most users of the SmartPad 
calculator are probably not sophisticated in mathematics, 
and sophisticated users may develop complex calculations . 
In either case, finding and fixing mistakes easily is 

10 important. For finding mistakes, the SmartPad calculator 
is inherently easy because the entire calculation is 
displayed and nothing is hidden from the user. This is 
better than spreadsheets, which display all the numbers but 
leave the formulas invisible (other than the selected one) . 

15 For correcting mistakes, the SmartPad calculator provides 
many options, most of which have a simple interface 
consistent with the scratch paper metaphor. 

Undo Function 

20 Most of the time, the user notices the mistake soon 

after making it. In these situations, the "undo" command 
makes correction easy. The user can undo virtually 
anything; even an "undo" can be undone with a "redo" 
command. The SmartPad calculator allows more than one past 

25 action to be undone. The number of undo-able past actions 
depends on the amount of RAM available. The undo and redo 
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CORRECTIONS 
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Everybody makes mistakes, so it should be easy to find 
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commands are always available. The user invokes these 
commands by tapping the mnemonics or icons displayed in the 
title area. 

5 Training Palette 

The "training palette" 30 of Figure 4 is used to 
correct mistakes by the handwriting recognizer , but not 
mistakes by the user. This allows the recognizer to learn 
from its mistakes and adapt to the user's handwriting. The 

10 training palette 30 is displayed in the title area of the 
screen 12 (instead of the title) when the user invokes the 
"learn" command. The palette 30 is a rectangular area with 
one box for each symbol (digit, operator, gesture, etc.) 
recognized by the handwriting recognizer. 

15 After writing something not recognized correctly, the 

user taps the stylus on the box with the correct symbol. 
The SmartPad calculator changes the symbol recognized to 
the symbol tapped and then adjusts its recognition process 
to do a better job next time. 

20 People usually write too fast to make corrections 

immediately, so the recognition error might not be noticed 
until after a few more characters are written. In that 
case, the user must select the incorrect symbol and then 
tap the correct one in the palette. The SmartPad 

25 calculator makes the correction, but the training feature 
would not be invoked if the original ink has been purged 
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from RAM. The SmartPad calculator normally retains the ink 
for the last several symbols. 

Users should "train" the SmartPad calculator when they 
first receive it, before trying their first real 
5 calculation. This avoids the distraction of correcting 
mistakes while thinking about the calculation. Training 
also ensures good recognition for symbols that are not used 
often. To train the SmartPad calculator , the user writes 
each symbol a few times and then corrects any that are 

10 recognized wrong. The training palette 30 shows the user 
which symbols to write. Alternatively , the SmartPad 
calculator may have a training mode in which it prompts the 
user to write the symbols. Training could also be part of 
a tutorial or a game that the SmartPad calculator provides 

15 for new users. When in training mode, there is also a way 
to "untrain" the SmartPad calculator, for someone using 
another person's SmartPad calculator. 

Corrections to Numbers 

20 Most user mistakes are incorrect numbers, especially 

when the number is copied from paper. Typical mistakes are 
incorrect digits, digits left out, and digits repeated, but 
sometimes the number is completely wrong. After fixing the 
number, the SmartPad calculator repeats all affected 

25 calculations and updates the screen 12 with the new 
results . 
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Figures 13A, 13B, and 13C illustrate how corrections 
are made on the screen 12 of the SmartPad calculator by 
overwriting the number. Assume, for example, that the 
number "7215" in Figure 13A should be the number "7265". 
5 To fix an incorrect digit, the user writes the correct 

digit over the incorrect one, as illustrated in Figure 13B. 
When writing on top of a character that already exists, the 
SmartPad calculator replaces the existing character with 
the new one, i.e., the number "7265", as illustrated in 

10 Figure 13C. Sometimes users try to "transform" the 

incorrect digit to the correct one. For example, to change* 
a "1" to a "4" the user might skip writing the vertical 
line, since the "1" already is a vertical line. The 
SmartPad calculator also recognizes corrections of this 

15 type. 

Figures 14A, 14B, 14C, 14D and 14E illustrate how 
corrections are made on the screen 12 of the SmartPad 
calculator by inserting digits in the number. Assume, for 
example, that the number "725" in Figure 14A should be the 

20 number "7265". To insert a digit that was left out, the 
number must first be "split open". In the SmartPad 
calculator, this is done by writing a caret "(5)" where the 
new digit is to be inserted, as illustrated in Figure 14B. 
The caret is a standard proofing mark to indicate 

25 insertion. Once the caret is recognized, the SmartPad 
calculator inserts a blank space where the caret was 
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written, as illustrated in Figure 14C. The user then 
writes the digit over the blank space as illustrated in 
Figure 14D. The SmartPad calculator replaces the blank 
space with digit, as illustrated in Figure 14E. To insert 
5 more than one digit, the user writes one caret for each 
digit and then writes the digits. 

Figures 15A, 15B and 15C illustrate how corrections 
are made on the screen 12 of the SmartPad calculator by 
deleting digits from the numbers. Assume, for example, 

10 that the number "72165" in Figure 15A should be the number 
"7265". To remove extra digits, the user draws a "delete" 
symbol over them, as illustrated in Figure 15B. The delete 
symbol is a line through the digits ending with a pigtail. 
This is a standard proofing mark for deletion. Once 

15 recognized, the SmartPad calculator removes the digits 

under the delete symbol and closes up the blank space, as 
illustrated in Figure 15C. Any number of digits can be 
removed with the delete symbol. 

Figures 16A, 16B and 16C illustrate how corrections 

20 are made on the screen 12 of the SmartPad calculator by 

interchanging digits in the numbers. Assume, for example, 
that the number "7625" in Figure 16A should be the number 
"7265". To interchange the digits, the user draws a 
"interchange" symbol over them, as illustrated in Figure 

25 16B. The interchange symbol is a "sine wave" separating 
the digits. This is a standard proofing mark for 



interchanging. Once recognized, the SmartPad calculator 
swaps the digits under the interchange symbol, as 
illustrated in Figure 16C. 

Figures 17A, 17B, 17C, 17D and 17E illustrate how 
5 corrections are made on the screen 12 of the SmartPad 

calculator by erasing digits from the numbers and replacing 
them with new digits. Assume, for example, that the number 
"72165" in Figure 17A should be the number "71625". To 
replace most or all of the number, the user draws an erase 

10 symbol over the digits, as illustrated in Figure 17B. The 
erase symbol is a line back and forth through the digits to, 
be erased, as if they were being scratched out. Unlike 
deletion, erasure does not close up the blank spaces, but 
instead leaves the area blank so new digits can be written, 

15 as illustrated in Figure 17C. The user then writes the 

digits over the blank space, as illustrated in Figure 17D. 
The SmartPad calculator replaces the blank spaces with the 
digits written, as illustrated in Figure 17E. Besides 
correcting numbers entered incorrectly, this method is used 

20 to experiment with different inputs to the calculations. 

Corrections To Calculations 

Sometimes the numbers are right but the calculation is 
wrong. Typical mistakes are incorrect operators, leaving 
25 something out, doing too much, doing the wrong thing, and 
doing things in the wrong order. After fixing the 



calculation, the SmartPad calculator computes the result, 
repeats any other affected calculations, and updates the 
display with the new results . 

Figures 18A, 18B and 18C illustrate how corrections 
5 are made on the screen 12 of the SmartPad calculator by 
overwriting the operator . Assume, for example, that the 
addition operator " + " in Figure 18A should be the 
multiplication operator "x" . To fix an incorrect operator, 
the user writes the correct operator over the incorrect 

10 one, as illustrated in Figure 18B. When writing on top of 
an operator that already exists, the SmartPad calculator 
replaces the existing operator with the new one, i.e., the 
multiplication. operator "x" , as illustrated in Figure 18C. 
Sometimes users try to "transform" the incorrect operator 

15 to the correct one. For example, to change a "-" to a "+" 
the user might skip writing the horizontal line, since the 
"-" already is a horizontal line. The SmartPad calculator 
also recognizes corrections of this type. 

Figures 19A, 19B, 19C, 19D, 19E and 19F illustrate how 

20 corrections are made on the screen 12 of the SmartPad 

calculator by inserting operators and numbers. Assume, for 
example, that the calculation "5 + 6 =" in Figure 19A 
should be the calculation "5 + 6 + 7 =". To add something 
left out of a calculation, the user first inserts blank 

25 space into the calculation, then writes in the new 

operators and numbers. The caret (which inserts digits in 
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numbers) is usually not appropriate since it does one 
character at a time. Instead , the user should select and 
move one part of the calculation away from the other, as 
illustrated in Figures 19B, 19C, and 19D. Once enough 
5 space is blank , the user writes in the new part of the 
calculation, as illustrated in Figure 19E. The SmartPad 
calculator recognizes when something new is added to a 
calculation by placement. If the new objects are written 
inside (or just before) the area occupied by the 

10 calculation, the SmartPad calculator inserts them into the 
calculation, as illustrated in Figures 19E and 19F. 

Figures 20A, 20B and 20C illustrate how corrections < 
are made on the screen 12 of the SmartPad calculator by 
deleting operators and numbers from the calculations . 

15 Assume, for example, that the calculation "5+6+7 =" in ( 
. Figure 2 OA should be the calculation "5 + 6 = To remove 
something from the calculation, the user deletes it with 
the delete symbol, as illustrated in Figure 2 OB. This is 
the same as removing digits from numbers. The delete 

20 symbol closes up the blank space, as illustrated in Figure 
20C. 

Figures 21A, 2 IB, 21C, 2 ID and 2 IE illustrate how 
corrections are made on the screen 12 of the SmartPad 
calculator by replacing operators and numbers in the 
25 calculations. Assume, for example, that the calculation "5 
+ 6 + 7 =" in Figure 21A should be the calculation "5 + 6 - 
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4 = ". To replace part of the calculation with something 
new, the user first erases the incorrect part, as 
illustrated in Figure 2 IB. The erase symbol leaves the 
blank space open so the user can write in the correct 
5 operators and numbers, as illustrated in Figure 21C. This 
is the same as replacing digits in numbers, where the erase 
symbol is a line back and forth through the operators and 
numbers to be erased, as if they were being scratched out. 
Unlike deletion, the erasure does not close up the blank 

10 space, as illustrated in Figure 21C. Instead, the SmartPad 
calculator leaves the area blank so that new operators and 
numbers can be written in, as illustrated in Figure 21D and 
2 IE. Besides correcting operators and numbers entered 
incorrectly, this method is also used to experiment with 

15 different operators and numbers in the calculations. To 
replace the entire calculation, the user should erase! it 
all. To stop the calculation without erasing it, the user 
can erase the result and/or the calculating symbol, as 
illustrated in Figures 22A, 22B and 22C. This leaves the 

20 expression without evaluating it. 

Figures 23A, 23B and 23C illustrate how corrections 
are made on the screen 12 of the SmartPad calculator by 
interchanging operators and numbers in the calculations. 
Assume, for example, that the calculation "5 + 6 + 7 =" in 

25 Figure 23A should be the calculation "5 + 7 + 6 =" . To 
interchange the operators and numbers, the user draws a 

56 



"interchange" symbol over them, as illustrated in Figure 
23B. The interchange symbol is a "sine wave" separating 
the operators and digits. This is a standard proofing mark 
for interchanging. Once recognized, the SmartPad 
5 calculator swaps the operators and numbers under the 
interchange symbol, as illustrate in Figure 23C. An 
alternative method of changing the order in a calculation 
is to select the part and then move it to the new position, 
as illustrated in Figures 24A, 24B, 24C, 24D, and 24E. 
10 There is no need to move more than once because the 

SmartPad calculator moves the rest of the calculation to 
maintain spacing alignment. 

Corrections to Links 

15 Users sometimes make mistakes in the overall logic of 

a calculation, especially if it is even moderately complex. 
A logic error means the calculations are not linked 
correctly. Typical mistakes include using the wrong result 
in another calculation, including an unnecessary 

20 calculation, or forgetting to include a necessary 

calculation. To fix these mistakes, the user must make 
changes that have a significant impact on the calculations. 
These changes are a little harder to do, so the user is 
less likely to accidentally make the wrong change. 

25 Most corrections to links require that a link be 

broken or that a link be established. The method for 
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breaking and establishing links depends on whether the link 
is by position, as illustrated in Figure 25A, or by 
copying, as illustrated in Figure 25B. A position link 
means the result of the first calculation happens to be 
5 placed inside the second calculation. A copy link means a 
copy calculation is used to copy the result of the first 
calculation to the second calculation. 

Figures 26A, 26B, 26C, 26D and 26E illustrate how 
position links are broken on the screen 12 of the SmartPad 

10 calculator. Figure 26A shows a horizontal calculation 

linked to a vertical calculation by a position. To break a 
position link, the user selects the result of the first 
calculation and moves it out of the second calculation, as 
illustrated by Figures 26B, 26C, and 26D. Since the result 

15 and its calculation must stay together, moving the result 
also moves its calculation, as illustrated in Figure 26D. 
Any calculations with a position link into the result * s 
calculation also move. To make links a little hard to 
break, the SmartPad calculator simulates a physical link by 

20 making the result appear to be stuck to the second 

calculation as the user tries to drag it out. However, if 
the user drags the stylus far enough away, the SmartPad 
calculator breaks the link. To show the user when the link 
is broken, the SmartPad calculator displays the result 

25 jumping out of the second calculation and to the current 
stylus position, as illustrated in Figure 26E. 
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Figures 27A, 27B and 27C illustrate how copy links are 
broken on the screen 12 of the SmartPad calculator* Figure 
27A shows two vertical calculations linked by a copy 
operation. To break a copy link, the user erases or 
5 deletes the copy (or perhaps the arrow that links the 
original to the copy) , as illustrated in Figures 27B and 
27C. 

Alternatively, the copy link can be broken by 
selecting and moving the copy out of the second calculation 

10 (the same as breaking a position link), as illustrated in 
Figures 28A, 28B, 28C, 28D, and 28E. This method of 
breaking the copy link does not delete the copy, so it is 
better if the copy is needed somewhere else. 

Figures 2 9A, 29B, 29C, 29D and 29E, illustrate how 

15 position links are made on the screen 12 of the SmartPad 
calculator. Figure 29A shows two separate calculations, 
one which is a horizontal calculation and the other which 
is a vertical calculation. To establish a position link 
between two existing calculations, the user selects the 

20 result of the first calculation and moves it into the 

second calculation, as illustrated in Figures 29B, 29C, and 
29D. There is no need to make room for the result in the 
second calculation, since the SmartPad calculator moves 
objects around to maintain alignment, as illustrated in 

25 Figure 29E. As with breaking links, the SmartPad 

calculator "resists" the effort to establish the link, 




making it hard to do accidentally. The result of the first 
calculation stops moving when it is close to the second 
calculation, as if the result cannot enter the calculation. 
However, if the user drags the stylus far enough inside the 
5 calculation, the SmartPad calculator establishes the link 
and the result jumps into the calculation. 

Figures. 30A, 30B and 30C illustrate how copy links are 
made on the screen 12 of the SmartPad calculator. Figure 
30A illustrates two separate calculations, both of them 

10 being vertical calculations. To establish a copy link 

between two existing calculations, the user creates a copy 
calculation from the result of the first calculation to the 
inside of the second calculation, as illustrated in Figures 
30B and 30C. Alternatively, if there is a copy of the 

15 result that is not being used, the copy can be selected and 
moved into the second calculation. This is the opposite of, 
breaking a copy connection by moving the copy out of the 
second calculation, as described above. 

Most link mistakes are fixed by a combination of 

20 breaking and establishing links. If the wrong calculation 
links into another calculation, the user breaks the link to 
the wrong calculation and then establishes the link to the 
correct one. If one of the calculations is unnecessary, 
the user breaks any links to the unnecessary calculation 

25 and then deletes the calculation. If there are any links 
left dangling after the deletion, the user deletes them or 



links them into the appropriate calculation. Finally, if 
an intermediate calculation is left out, the user breaks 
the link in the main calculation where the intermediate 
calculation is to go and then establishes the links into 
5 and out of the intermediate calculation. If necessary, 
some calculations may be moved first to make room for the 
intermediate calculation. 

Corrections to Labels 

10 Figures 31A, 31B, 31C and 31D illustrate how label 

corrections are made on the screen 12 of the SmartPad 
calculator. Figure 31A shows a label "wrong". When a 
label is incorrect, the user first erases it and then 
writes a new one, as illustrated in Figures 31B, 31C and 

15 31D. Writing over an existing label does not fix it since 
the new writing is added to the original; this behavior is 
needed so users can dot "i's" and cross "t f s". To erase a 
label, the erase or delete symbol is drawn over the label. 

20 13. ADJUSTMENTS 

With scratch paper, calculations often end up laid out 
poorly. For example, part of the calculation may be too 
close to the edge of the page, or two calculations may be 
too close together, or the whole calculation may be too big 

25 to fit on the page. To fix these problems on scratch 

paper, the calculations must be erased and rewritten. With 
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the SmartPad calculator, objects on the screen 12 are 
easily moved around, re-sized, rotated, and scrolled. 
These adjustments can be done at any time. They have no 
effect on the calculations, only on the way they are 
5 displayed. 

Moving Objects 

Figures 32A, 32B, 32C, 32D, 32E and 32F illustrate how 
object position changes are made on the screen 12 of the 

10 SmartPad calculator. Figure 32A shows two objects, i.e., 
the numbers "628" and "91", positioned on opposite sides of 
screen 12. To change the position of a SmartPad calculator 
object, the user first selects it with the lasso, as 
illustrated in Figure 32B. Once selected, the user touches 

15 the stylus tip anywhere inside the selection, as indicated 
by the handles in Figure 32C. To show that the selection 
can be moved, the SmartPad calculator displays a hand under 
the stylus, as illustrated in Figure 32D. The user "drags" 
the stylus to a different place, as illustrated in Figure 

20 32E. The hand and selection follow the stylus as if 
attached to it. Moving stops when the user lifts the 
stylus from the screen, as illustrated in Figure 32F. The 
SmartPad calculator does not allow two objects to occupy 
the same space. If the user drags the stylus over another 

25 object, the selected object will stay away. However, the 
user can (with some "effort") move a number or operator 
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into a horizontal or vertical calculation; this makes the 
object part of the calculation. Similarly, effort is 
required to move an object out of a calculation. 

Any number of objects can be moved in one drag by 
5 selecting all of them first. Each object is moved the same 
amount, so they maintain the same relative positions. 
Since objects cannot overlap, multi-object selections have 
more constraints on the places they can move to. If 
necessary, other objects must be moved out of the way 
10 first. Multi-object selections cannot be moved into 

calculations unless all of them fit and can be part of the 
calculation. 

Since results of horizontal and vertical calculations 
have fixed positions relative- to their formulas, moving a 

15 result also moves the other objects in the calculation. If 
any numbers in the calculation are the results of other 
horizontal or vertical calculations, then all objects in 
the other calculations move as well. This makes it 
possible to move numerous objects by just moving one 

20 result, as illustrated in Figures 33A, 33B, and 33C. 

With copy operations, moving the original or copy does 
not affect the position of the other one. The SmartPad 
calculator updates the arrow to point from the original to 
the copy, as illustrated in Figures 34A, 34B, and 34C. 

25 
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Changing Size 

Figures 35A, 35B, 35C, 35D, 35E and 35F illustrate how 
changes to the size of objects are made on the screen 12 of 
the SmartPad calculator. Assume , for example, that the 
5 number "628" in Figure 35A should be enlarged for 

readability. To change the size of a SmartPad calculator 
object, the user first selects it with the lasso, as 
illustrated in Figure 35B. The SmartPad calculator 
displays four small squares (handles) around" the selection, 

10 as illustrated in Figure 35G. When the stylus tip touches 
a handle, the SmartPad calculator displays a hand grabbing 
the handle, as illustrated in Figure 35D. When the stylus 
is dragged across the screen, the hand holding the handle 
stays with it, except that it must remain in line with th<e 

15 original handle. position and the handle opposite it, as 
illustrated in Figure 35E. The two adjacent handles also 
move to maintain the rectangular arrangement. When the 
user lifts the stylus, the selected object is re-sized to 
fit inside the new handle positions, as illustrated in 

20 Figure 35F. The handle opposite the one grabbed does not 
move, so the user chooses the direction to expand to by 
choosing the handle in that direction. Objects can be 
increased or decreased to any size (within reason) . 
However, the size cannot be increased to the point that 

25 other objects are overwritten. Any number of objects can 
be re-sized in one drag by selecting all of them first. 
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Each object is re-sized the same amount, so they maintain 
the same relative sizes. In addition, the objects move 
apart the same amount, so they maintain the same relative 
positions and do not overlap. 
5 All objects in a single horizontal or vertical 

calculation must have the same size, except that the result 
can be different. Thus, changing the size of any number or 
operator in a calculation causes the others to change in 
size as well (except for the result) . In a copy operation, 
10 the original and the copy can be re-sized independently. 

Changing Orientation 

Figures 36A, 36B, 36C and 36D illustrate how changes 
are made to the orientation of the screen . 12 of the 

15 SmartPad calculator. The SmartPad calculator display 

usually has one side longer than the other, as illustrated 
in Figure 36A. Horizontal calculations fit better with the 
long side horizontal, while vertical calculations prefer 
the long side vertical. In addition, right-handed users 

20 work better with the stylus attached on the right side, 
while left-handed users prefer the left side. To 
accommodate these situations, the SmartPad calculator 
allows any one of the four sides of the screen 12 to be 
"top". To change the orientation, the user draws an arrow 

25 from the middle of the current top edge to the middle of 
the new top edge, as illustrated in Figure 36B. All 





objects on the page must have the same orientation, but 
each page in the pad has its own orientation. When the 
user goes to another page, the screen 12 rotates to that 
page's orientation. 



scrolling is performed on the screen 12 of the SmartPad 
calculator. The SmartPad calculator supports pages of 

10 unlimited size. Rather than making objects smaller to fit 
in the screen 12, the user can "scroll" the screen 12 to 
show a different area of the page. Before a page can be 
scrolled, the user must draw a "compass", which is a star- 
like figure that. remains attached to the page, as 

15 illustrated in Figure 37A. The SmartPad calculator 
responds by displaying a formal compass icon at the 
indicated location on the screen 12, as illustrated in 
Figure 37B. To scroll the screen 12 over the page, the 
user touches the compass with the stylus, and then drags it 

20 to a different place on the screen 12, as illustrated in 
Figures 37C and 37D. The compass (and with it the page) 
behaves as if attached to the stylus. The user can drag 
the compass anywhere on the screen 12. If more scrolling 
is required after the compass reaches the edge of the 

25 screen 12, then a second compass can be drawn and dragged. 
There is, no limit on the number of compasses on a page. A 
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Scrolling 



Figures 37A, 37B, 37C, and 37D illustrate how 
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compass need only be created once / since it remains until 
the user deletes it. Like other SmartPad calculator 
objects , a compass can be selected and moved to a different 
place on the page, as long as it does not overlap other 
5 objects. 

14 . PAGES 

Small versions of the SmartPad calculator may have 
only one page, but many users want pads with multiple 

10 pages. With more than one page, users can keep unrelated 
calculations on different pages. To support multiple 
pages, the SmartPad calculator has ways of identifying, 
accessing, creating, deleting, splitting^ copying, merging, 
and sorting pages. These features also allow the user to 

15 move and copy calculations from one page to another. The 
screen always displays one page at a time. 

Identification and Access 

The SmartPad calculator identifies pages by number and 
20 by title. The SmartPad calculator handles page numbers 
automatically, wherein pages are numbered consecutively 
from one. The user writes in page titles and the SmartPad 
calculator treats the title as a label. The number and 
title of the current page are displayed in the "title area" 
25 at the top of the screen 12, as illustrated in Figure 3. 

The title area also displays a menu, shown as a column 
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on the right side of the screen 12 in Figure 3, which 
includes the undo and the train or redo commands. When 
training , the training palette is displayed instead of the 
page number and title. To change the title , the user 
5 erases the current title and then writes a new one (the 
same as with labels). To view another page, the user 
writes the new page number over the, current page number. 
If the number of the other page is not known, the user 
displays page zero, which has the "table of contents". 

10 Figure 38 shows a table of contents, which lists pages 

by number and title. At the top of Figure 38 is displayed 
the title of the pad, and on the right is a command menu 
and scroll bar. The user touches the scroll bar to view 
pages that are not visible on the screen 12. To select a 

15 page, the user taps the page number or title once and the 
selected pages are highlighted by flashing the page number. 
To go to any page, the user taps the page number or title 
twice. Finally, the user can follow an inter-page link by 
tapping a link object twice. The SmartPad calculator goes 

20 to the other page with that link. 

Creation and Deletion 

Every pad has an extra blank page listed at the end of 
the table of contents, as illustrated in Figure 38. To 
25 create a new page, the user goes to the blank page and 

starts writing in it. As soon as that page has writing on 





it, the SmartPad calculator creates another blank page 
after it. New blank pages are always available (until the 
SmartPad calculator is out of RAM) . 

To remove a page from the pad, the user draws an erase 
5 or delete symbol over the page number in the table of 

contents . The SmartPad calculator asks the user to verify 
that action, then removes the page from the pad and 
discards all calculations on that page. Unless the user 
immediately taps "undo", the calculations are permanently 
10 lost. 

Splitting, Copying, and Merging 

Figures 39A, 39B, 39C, and 39D illustrate how pages 
can be split on the screen 12 of the SmartPad calculator. 

15 At some point. when developing a large ■ set of calculations, 
one page is not enough. To split up one page's 
calculations into two pages, the user goes to the page and 
selects everything to place in the second page, as 
illustrated in Figures 39A and 39B. Then, the user goes to 

20 the table of contents and taps the "split" command in the 
menu. The SmartPad calculator creates a new page and 
transfers all selected objects to it, as illustrated in 
Figures 39C and 39D. The new page is untitled and follows 
the original page in numbering, and the SmartPad calculator 

25 displays the new page. If the splitting causes a result to 
be separated from its formula, the result is duplicated on 
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both pages and an inter-page link is created between them, 
as illustrated in Figures 39C and 39D: 

Copying calculations is similar to splitting them off. 
To make a copy of some calculations on a page, the user 
5 selects everything to copy, goes to the table of contents, 
and taps the "copy" command. The SmartPad calculator 
copies all selected objects to the new page, but without 
removing them from the original page. To copy an entire 
page, the user need not go to the page and select 

10 everything on the page. Instead, the user goes to the 

table of contents, selects the page to copy, and then taps, 
the "copy" command. 

The opposite of splitting is merging. To merge the 
calculations on two separate pages into a single page, the 

15 user goes to the second page to make it the current page 
and then goes to the table of contents. Then, the user 
selects the first page and taps the "merge" command. The 
SmartPad calculator copies everything from the second page 
to the first page and then deletes the second page. The 

20 SmartPad calculator displays the first page and 

automatically selects everything that came from the second 
page. The user then moves the selected calculations to the 
desired position relative to the calculations from the 
first page. If there were inter-page links between the two 

25 original pages, the SmartPad calculator replaces then with 
direct copy calculations. To transfer calculations from 
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one page to another, the user first splits them off , then 
merges them with the other page. 

Reordering 

5 To change the order of the pages in the pad, the user 

first goes to the table of contents and selects a page to 
move. When the stylus touches the selection, a "hand" is 
drawn to show it can be moved. The user drags the hand 
(and page, which behaves as if attached) to a new position 
10 in the table of contents. After the stylus is lifted from 
the screen, the SmartPad calculator moves the selected page 
to the new position and automatically renumbers every page. 

15. PADS 

15 Some SmartPad calculators support removable memory 

(e.g., magnetic cards or floppy disks). With removable 
memory, users can save multiple pads, or multiple copies of 
one pad. Any number of pads can be saved (until out of 
memory). To support multiple pads, the SmartPad calculator 

20 has ways of identifying, accessing, creating, deleting, 

splitting, copying, and merging pads. These features also 
allow users to transfer and copy pages from one pad to 
another. In addition, the SmartPad calculator can transfer 
pads from one device to another, and can restrict access to 

25 pads. 
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Identification And Access 

Pads are identified by a title, which is displayed in 
the title area of the table of contents, as illustrated in 
Figure 38. Like page titles, the user writes the pad title 
5 directly into the title area. Since pads are easily 

copied, the SmartPad calculator also identifies pads by a 
copy and change number. Copy numbers are unique for each 
copy, while change numbers are the same unless something in 
the pad has been changed. Copy and change numbers always 
10 increase, so the newer copy can always be determined. 

Finally, the SmartPad calculator also includes a clock, so 
that the date and time of the last change can be recorded 
in each pad. 

To access a saved pad, the user inserts the removable 
15 memory into the slot. The SmartPad calculator displays a 
table of contents (file listing) for the memory. The 
listing shows the title, copy number, and date or change 
number of each saved pad. In addition, on top is a title 
area for identifying the removable memory and at the right 
20 is a command menu. To select a saved pad, the user taps 
its listing. 

To save the current pad onto removable memory, the 
user taps the "save" command. The SmartPad calculator 
warns the user of anything unusual (e.g., the pad has 
25 already been saved) . If there is no room, the SmartPad 
calculator asks if older copies can be deleted first. To 
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load a pad from removable memory (making it the current 
one), the user selects the pad and then taps the "load" 
command. If the current pad has unsaved changes, the 
SmartPad calculator saves it first. The SmartPad 
5 calculator warns of anything unusual (for example, not 
loading the most recently saved copy) . If the pad was 
created on a different device, the SmartPad calculator 
checks if the user is the author. If so, the author's 
training for the handwriting recognizer is loaded as well, 
10 so the new device need not be retrained. 

Creation and Deletion 

To create a new (empty) pad, the user inserts the 
removable memory and then taps the "new" command. If 

15 necessary, the SmartPad calculator saves the current pad 

first. To create a new pad on the SmartPad calculator, the 
user can also delete every page of the current pad and 
change its title. To delete a pad in removable memory, the 
user selects the pad and then taps the "remove" command. 

20 The SmartPad calculator asks the user to verify the action, 
since the pad is permanently lost after deletion. 

Splitting, Copying, and Merging 

To split one pad into two, the user inserts the 
25 removable memory and loads the pad to split. The user then 
selects the pages that are going into the second pad. 
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Finally, the user goes back to the pad listing and taps the 
"split" command. The SmartPad calculator saves the 
original pad without the selected pages, then removes the 
unselected pages from the current pad, making it the new 
5 one. The user must change the title of the current pad to 
distinguish it from the original. The SmartPad calculator 
deletes any inter-page links between the two pads. Copies 
of the numbers appear on both pads but are not linked. 

Copying pages out of pad is similar to splitting. The 

10 user selects the pages to copy, goes to the pad listing, 
and taps the "copy ,, command. The original pad is saved 
without making any changes, and the unselected pages are 
removed from the new (current) pad. To copy an entire pad, 
the user need not go to the pad and select all pages. 

15 Instead, the user selects the pad in the pad listing and 
then taps the "copy" command. 

To merge two pads into one, the user inserts the 
removable memory and loads the second pad. The user then 
goes to the pad listing, selects the first pad, and taps 

20 the "merge" command. The SmartPad calculator reads in the 
selected pad, adding its pages to the current pad. The 
SmartPad calculator then goes back to the table of contents 
for the current pad, so the user can place the pages in the 
desired order. 

25 To transfer or copy pages from one pad to another, the 

user first splits off those pages from the source pad, then 
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merges them with the destination pad. 
Porting 

File formats of the SmartPad calculator are compatible 
5 across all platforms. Users can easily transfer pads form 
one device to another, as long as both support the same 
kind of removable memory or data interface. To transfer a 
pad by way of removable memory, the user inserts the memory 
in the source device and saves the pad. The memory is then 

10 removed and inserted in the destination device, where the 
pad is loaded. To transfer a pad through the data 
interface, the two devices are connected (with infrared 
transceivers, the devices are pointed at each other). Then 
the "receive" command is selected on the destination 

15 device, followed by the "send" command on the source 

device. After the transfer is complete, the devices are 
disconnected. 

Security 

20 Users with sensitive information in a pad want a way 

f ■ - 

to prevent others from viewing the pad. Sometimes, the 
author of a pad is willing to have the pad viewed by 
others, but not changed. To secure access to a pad, the 
user goes to the table of contents and taps the "secure" 
25 command. The SmartPad calculator displays the "security" 
page, where the user chooses the level of security and the 
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access method. The four levels of security are: total, 
view, input, and none. Total security means the pad cannot 
be seen without following the access protocol. View 
security means the pad can be viewed, but cannot be changed 
5 without proper access. Input security means the pad can be 
viewed and the input numbers can be changed, but changes to 
the layout or the calculations require proper access. In 
all cases, changes to security require proper access. 
There may also be restrictions on deleting or duplicating 

10 the pads. 

The SmartPad calculator might offer one or more of 
three access methods: passnumber , password, or signature. 
The passnumber method requires the user to write in the 
author's "secret" number to gain access. The password 

15 method presents the user with a grid of letters, wherein 
the user must tap the letters in the right order to spell 
out the author's secret password. Finally, the signature 
method requires the user to write the author's signature, 
wherein the SmartPad calculator uses handwriting 

20 recognition to verify the signature. To make forgery 

difficult, the signature must be written quickly. To make 
sure the SmartPad calculator can recognize a valid 
signature, the author at some point must train the SmartPad 
calculator by repeatedly signing until the signature is 

25 recognized most of the time. 
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16. PRINTING 

Printed copies of the SmartPad calculator calculations 
provide a permanent record for backup or archiving, and 
allow calculations to be communicated to others. Unless 
5 the screen 12 can be photocopied, printing requires either 
a built-in printer or an attached printer compatible with 
the data interface. Another option is to transfer the pad 
(by way of removable memory) to another SmartPad calculator 
equipped with a printer. A SmartPad calculator on a 
10 general-purpose computer provides the most printing 
options . 

Printers with fully addressable pixels offer the best 
printouts, since the text of the calculations can be placed 
anywhere, and the graphic labels can be printed as well. 

15 With these printers, the hard copy is a replica of the page 
as displayed on the screen 12. If the real paper is 
smaller than the screen 12 (or page), the SmartPad 
calculator reduces the size of the image to fit the paper. 
The SmartPad calculator might also support printing on 

20 text-only printers. The SmartPad calculator makes the 

printout resemble the screen 12 as much as practical, given 
the limits of the printer. This includes placing numbers 
and symbols to maintain position relationships, and drawing 
lines as a series of characters . 

25 To print a page of calculations, the user goes to the 

table of contents and selects the page to print, then taps 
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the "print" command. If a printer is built into the 
platform, the SmartPad calculator sends the image to that 
printer. Otherwise, the image is sent out the data 
interface. 

5 

17. EXPORTING 
Very complex calculations may exceed the SmartPad 
calculator's capabilities or degrade its performance. 
Users can "export" these calculations to a more capable 
10 application, such as a spreadsheet or a programming 
language . 

To Spreadsheet 

A spreadsheet is a rectangular grid of cells, wherein 

15 each cell is either blank or contains a label, a number, or 
a formula and the numeric result of evaluating that 
formula. The main differences between the SmartPad 
calculator and a spreadsheet are: (1) spreadsheet cells 
lie on grid but the SmartPad calculator numbers can be 

20 placed anywhere; (2) formulas in spreadsheets occupy the 
same cell as the numeric result, but the SmartPad 
calculator formulas are next to the results; (3) and 
spreadsheet labels are text, but the SmartPad calculator 
labels are graphics. 

25 To export a calculation to a spreadsheet, the user 

first goes to the pad's table of contents and selects the 
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page to be exported. The user then taps the "spreadsheet" 
command. The SmartPad calculator creates a spreadsheet 
file equivalent to the selected page. If the user selects 
another page and exports it, the two pages are merged into 
5 the same spreadsheet file. This allows inter-page link to 
be resolved within the spreadsheet. When a SmartPad 
calculator calculation is converted to a spreadsheet, every 
number in the calculation is placed in one of the cells of 
the spreadsheet. The SmartPad calculator picks the cells 

10 to maintain the position relationships between the numbers. 
After that, formulas are generated for every result cell. 
Finally, any other required information (such as number 
cell formats) is generated and the file is output. The 
file is output in an industry- standard format. 

15 If the SmartPad calculator platform has a keyboard, 

the user can replace the ink labels with real text. The 
SmartPad calculator could then export the labels to the 
spreadsheet. As with numbers, the SmartPad calculator 
picks cells to hold the labels to maintain position 

20 relationships. 

Programming 

A program is an algorithm expressed in a programming 
language, wherein the algorithm specifies changes in the 
25 values that are stored in variables. The main differences 
between the SmartPad calculator and a programming language 
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are: (1) numbers in a program must be stored in named 
variables, but the SmartPad calculator numbers are stored 
at unnamed positions on the display; and (2) a program has 
no labels other than the variable identifiers, which lack 
5 full expressiveness. 

To export a calculation to a program, the user does 
the same thing as exporting to a spreadsheet, except the 
"program" command is tapped instead of the "spreadsheet" 
command. In addition, the user is expected to provide 

10 variable names for every number, which makes a keyboard 
highly desirable. 

When a SmartPad calculator calculation is converted to 
a program, a variable is created for each number, wherein 
the names of the variables are provided by the user. After 

15 that, assignment statements for each variable are generated 
in the appropriate order. Finally, the file is output as 
plain text, suitable for input to any compiler. The 
program is generated in the C programming languages; other 
languages may also be supported. 

20 

18. PLOTTING 

The SmartPad calculator might support plotting. Plots 
can either be continuous or empirical . A continuous plot 
is a curve showing the result of a calculation as a 
25 function of an input, wherein the user identifies which 

number is the input and which is the result. An empirical 

80 



plot is a series of bars or dots showing the values from a 
list or table, wherein the user enters the values in a 
table, or generates by a calculation. 

Using the SmartPad calculator, the user draws the axes 
5 of the plot, e.g., by drawing a big "L" character. In 
response, the SmartPad calculator re-draws the axes and 
formalizes them by displaying a rectangle graphic. To 
indicate the plotting range, the user writes numbers at the 
axes ■ ends and the SmartPad calculator fills in 
10 intermediate values on the axes. Thereafter, the SmartPad 
calculator draws the plot by adding lines or dots to the 
screen 12 . 

For continuous plots, the user first identifies 
variables for the plot by drawing copy operation arrows. 
15 The SmartPad calculator then performs the plot generation, 
wherein it evaluates each point. Finally, the SmartPad 
calculator displays the plot by drawing lines through the 
points of the plot. 

For empirical plots, the user first tabulates the 
20 desired plot values by creating a list of numbers on the 
screen 12 of the SmartPad calculator. The SmartPad 
calculator then displays the plot by drawing lines, dots or 
bars through the points of the plot. 

For plotting operations, the SmartPad calculator also 
25 supports such "cleanup" functions as re-sizing the plot, 
changing the range of the plot, changing the calculations, 
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and changing the data displayed on the plot. 

19 .TEACHING 

A teaching version of the SmartPad calculator may 
5 provide instruction and practice in manual arithmetic to 
primary students. The screen 12 displays problems in 
elementary arithmetic to the student. The student solves 
the problem manually by writing on the screen 12, as 
students do now on paper. Every step of the manual process 

10 is recognized by the teaching SmartPad calculator. The 
teaching SmartPad calculator notices when a student is 
doing something wrong and provides immediate feedback. 

A teacher interface portion for the SmartPad 
calculator allows the teacher to set problem levels, 

15 generate problems, and l review the student's work on the 

problems . A student interface for the SmartPad calculator 
allows the student to display the problems, display on-line 
tutorials or instruction, respond to the problems, etc. 

20 20. CHECKBOOK 

A checkbook version of the SmartPad calculator 

simulates a manual checkbook register, using handwriting 

recognition to automate many functions. The checkbook 

version of the SmartPad calculator holds the checks, and 

25 automatically turns on when the checkbook is opened. 

Checks are inscribed on the screen 12 of the SmartPad 
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calculator using a real pen instead of a stylus. The 
screen 12 of the SmartPad calculator acts as a "carbon 
copy" for the information written on the check, wherein the 
SmartPad calculator senses the position of the pen on the 
5 checks . The amount and date on the check are recognized by 
the SmartPad calculator and are entered into a check 
register therein. The name of the recipient is not 
recognized, but is entered into the check register as 
written. The SmartPad calculator automatically updates the 

10 account balance, displays the current date, shows a 

calendar, does simple calculations, reminds the user when 
regular payments are due, permits manual corrections, and 
issue warnings if something looks wrong (e.g., incorrect 
date). Finally, the SmartPad calculator allows the check 

15 register to be printed. 

21. DATA STRUCTURES 
The processing circuit requires inputs, generates 
outputs, requires constants, and uses variables to control 
20 the operation of the SmartPad calculator. Some of this 
data is described in more detail below. Those skilled in 
the art will recognize that other forms of data could be 
used as well. 

25 
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Inputs 

current time 
screen turned on or off 
pen touching screen or not 
5 pen position 

Outputs 

display image (bitmap of screen) 

ink image (bitmap of written ink; overlays display 
10 image) 

Constants 

screen width and height 
page area height (at bottom of screen) 
15 ' title/palette width and height (at top left corner of 
screen) 

number of palette rows and columns 
palette column width and row height 
for each palette row 
20 for each palette column 

palette symbol 

symbol image (bitmap of symbol displayed on 
screen) 

menu width and height (at top right corner of screen) 
25 number of menu rows and columns 

menu column width and row height 
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for each menu row 

for each menu column 
, menu item image (bitmap of menu item 

displayed on screen) 
5 blink duration (for blinking selected menu items) 

flash on and off duration (for flashing selected 
objects) 

animate on and off duration (for animating 
calculations ) 
10 character form factor (width/height ratio) 

counter form factor (width/height ratio) 
result line form factor (height) 
arrowhead form factor (width and height) 
scroller form factor (inner and outer size) 
15 selection handle size 

caret width and height 
for each symbol : 

represents action (something to do) 
or object (something to create) 
20 default model (for handwriting recognition) 

Variables 

displaying title or palette 

scroll position (point displayed at top left corner of 
25 page area) 

initial pen position (where pen first touched screen, 
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last time it did) 
ink written (list of points defining path of pen, last 

time pen touched screen) 
symbol written: none, character A-Z, digit 0 to 9, 

period, plus, minus, times, divide, left 

parenthesis, right parenthesis, line, arrow, 

compass, frame, scratch-out, pigtail, caret 

(horizontal, vertical), wave (horizontal, 

vertical) 
for each symbol: 

model (for handwriting recognition; this 
includes training to date) 
list of objects on page: 

kind of object: label, scroller, character, 

result line, arrow, number, counter, and 
calculation (horizontal, vertical, copy) 

original ink (list of points; copy of written ink 
creating this object; not for numbers or 
calculations; may be purged when out of 
memory) 

selected or not (not for calculations) 

area (rectangular region this object occupies; 

not for arrows) 
graphic form (list of points; stylized 

representation of this object; not for 
characters, numbers, or calculations) 
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moved or not (if moved, nearby objects need to be 

aligned at next available opportunity; not 

for calculations) 
drawn or not (if not drawn, object needs to be 
5 drawn at next available opportunity; not for 

calculations) 
children (list of component objects; only for 

numbers, counters, and calculations; numbers 

and counters only contain characters; 
10 calculations contain characters, numbers, 

counters, result lines, and arrows) 
parent (number, counter, or calculation that 

includes this object as a child; not for 

labels, scrollers, or calculations) 
15 calculator (calculation that sets this object's 

value; only for numbers) 
list of copiers (copy calculations that use this 

object's value; only for numbers and 

counters ) 

20 value (only for characters, numbers, and 

counters ) 

computed or not (if not computed, value needs to 
be computed at next available opportunity; 
only for numbers ) 
25 - list of labels in title: 

graphic form (list of points) 
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animated calculation (which calculation is currently 

being animated) 
animated object (which object in animated calculation 

is being flashed) 
5 animate on or off (animated object visible or not) 

animate time (time of next change to visibility) 
selection area (rectangular region enclosing all 

selected objects on page) 
flash on or off (selected objects visible or not) 
10 flash time (time of next change to visibility) 

handles displayed or not 

undo page list (copy of list of objects on page, to 
allow undo) 

undo title list (copy of list of labels in title) 
15 undo selection area (copy of selection area) 

22. FLOW CHARTS 
Figures 40-58 are flow charts describing the logic 
performed by the processing circuit in the SmartPad 

20 calculator. Those skilled in the art will recognize that 
this logic may be implemented in either hardware or 
software. Those skilled in the art will also recognize 
that the logic could be altered without departing from the 
scope of the present invention. 

25 Figure 40 is a flow chart describing the logic of the 

Program function in the preferred embodiment of the 
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SmartPad calculator. The Program function provides the 
main processing loop for the SmartPad calculator. Block 
4002 represents the start of the Program function. Block 
4004 performs the Initialize function, as described in more 
5 detail in Figure 41. Block 4006 performs the Update-Page 
function, described in more detail in Figure 42. Block 
4008 is a decision block that determines if the screen is 
turned on. If the screen is not turned on in Block 4008, 
control is transferred back to Block 4006. If the screen 

10 is turned on in Block 4008, then control is transferred to 
Block 4010. Block 4010 is a decision block that determines 
whether the pen or stylus is touching the screen. If the 
pen is not touching the screen, then control Is transferred 
to Block 4006. If the pen is touching the screen, then 

15 control is transferred to Block 4012. Block 4012 performs 
the Respond-to-Pen function, as described in more detail in 
Figure 43. 

Figure 41 is a flow chart describing the logic of the 
Initialize function in the SmartPad calculator. Block 4102 

20 represents the start of the Initialize function. Block 

4104 clears the display image structure. Block 4106 clears 
the ink image structure. Block 4108 draws the title on the 
screen. Block 4110 draws the menu on the screen. Block 
4112 sets a status flag which indicates that the title has 

25 been displayed. Block 4114 resets the scroll position of 
the screen, i.e., the point displayed at the top left 



corner of the page area on the screen. Block 4116 clears 
the ink written structure. Blocks 4118 - 4124 loop to set 
all Symbols to a Default Model. Block 4126 sets the list 
of objects on the page to "none". Block 4128 sets the list 
5 of labels in the title to "none". Block 4130 sets animate 
calculation to "none". Block 4132 sets the Selection Area 
to "none". Block 4134 sets the undo page list, undo title 
list, and undo selection area to "none". Block 4136 
represents the termination of the Initialize function. 

10 Figure 42 is a flow chart describing the logic of the 

Update-Page function in the SmartPad calculator. Block 
4202 represents the start of the Update-Page function. 
Block 4204 finds the first object on the page that has 
moved. Block 4206 is a decision block that determines 

15 whether any object has moved on the page. If an object has 
moved, then control is transferred to Block 4208. Block 
4208 aligns any related objects and control is returned at 
Block 4210. If Decision Block 4206 indicates that an 
object has not moved, then Block 4212 finds the first 

20 object on the page that is not drawn. Block 4214 is a 

decision block that determines whether there is any object 
not drawn. If an object is not drawn, then control is 
transferred to Block 4216. Block 4216 displays the object 
on the screen. Control is then transferred to Block 4210, 

25 which terminates the Update-Page function. If Decision 

Block 4214 indicates that all the objects have been drawn, 
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then control is transferred to Block 4218. Block 4218 is a 
decision block that determines whether there is a selection 
area. If there is a selection area, then control is 
transferred to Block 4220. Block 4220 is a decision block 
5 that determines if handles are displayed for the selection. 
If handles are not displayed, then control is transferred 
to Block 4222. Block 4222 inverts the selection handles on 
the screen, and control is transferred to Block 4210. 
Block 4210 terminates the Update-Page function. If Block 

10 4218 indicates that there is no selection area, or if Block 
4220 indicates that handles are displayed, then control is 
transferred to Block 4224. Block 4224 is a decision block 
that determines whether there is an animated calculation. 
If there is an animated calculation, then control is 

15 transferred to Block 4226. Block 4226 is a decision block 
that determines whether the current time is later than the 
animate time. If so, then control is transferred to Block 
4228. Block 4228 animates the calculation, and then 
control is transferred to Block 4230. Block 4230 

20 terminates the Update-Page function. If Decision Block 

4226 indicates that the current time is not later than the 
animate time, then control is transferred to Block 4230. 
Block 4230 then terminates the Update-Page function. If 
Decision Block 4224 indicates that there is no animated 

25 calculation, then control is transferred to Block 4232. 

Block 4232 finds a number ready to calculate, by searching 
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every object on the page that is a number, where the number 
has not been computed and the object has a calculator 
symbol. Block 4234 is a decision block that determines 
whether there is a number ready to calculate. If so, then 
5 control is transferred to Block 4236 which performs the 
Calculate function, as described in more detail in fi g ur e 
-5€. After the Calculate function is performed, control is 
transferred to Block 4230 and the Update-Page function is 
terminated. If Decision Block 4234 indicates that there is 

10 no number ready for calculation, then control is 

transferred to Block 4238. Block 4238 is a decision block 
that determines whether there is a selection area. If so, 
then control is transferred to Block 4240. Block 4240 is a 
decision block that determines whether the current time is 

15 later than the flash time. If Decision Block 4240 

indicates that the current time is later than the flash 
time, then control is transferred to Block 4242. Block 
4242 flashes the selection area on the screen. Control is 
then transferred to Block 4230 and the Update-Page function 

20 is terminated. If Decision Block 4238 indicates that there 
is no selection area, then control is transferred to Block 
4230 and the Update-Page function is terminated. 
Similarly, if Decision Block 4240 indicates the current 
time is not later than the flash time, then control is 

25 transferred to Block 4230 and the Update-Page function 
terminates . 
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Figure 43 is a flow chart describing the logic of the 
Respond-to-Pen function in the SmartPad calculator. Block 
4302 represents the start of the Respond-to-Pen function. 
Block 4304 reads the initial pen position on the screen. 
5 Block 4306 is a decision block that determines whether the 
pen is in the title or the palette area of the screen. If 
so, the control is transferred to Block 4308. Block 4308 
is a decision block that determines whether the title is 
displayed. If so, then control is transferred to Block 

10 4310. Block 4310 performs the Handle-Pen-In-Title 

function, as described further in Figure 44. Control is 
then transferred to Block 4312 which erases the ink image. 
Block 4314 represents the termination of the Respond-to-Pen 
function. If Decision Block 4308 determines that the title 

15 is not displayed, then control is transferred to Block 
4316. Block 4316 performs the Handle-Pen-In-Palette 
function, as described in more detail in Figures 45A and 
45B. Control is then transferred to Block 4312 which 
erases the ink image. Block 4314 terminates the Respond- 

20 to-Pen function. If Decision Block 4306 indicates that the 
pen is not in the title or palette area of the screen, then 
control is transferred to Block 4318. Block 4318 is a 
decision block that determines whether the pen is in the 
menu area of the screen. If Decision Block 4318 determines 

25 that the pen is in the menu area of the screen, then 

control is transferred to Block 4320. Block 4230 performs 
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the Handle-Pen-In-Menu function, as described in more 
detail inV Figurc 4 6 . After this function is performed, 
control is transferred to Block 4312 which erases the ink 
image* Block 4314 represents the termination of the 
5 Respond- to-Pen function. If Decision Block 4318 indicates 
the pen is not in the menu area of the screen, then control 
is transferred to Block 4322. Block 4322 performs the 
Handle-Pen-In-Page function, as described in more detail in 
Figures 47A and 47B. After this function is performed, 

10 control is transferred to Block 4312 which erases the ink 
image. Block 4314 represents the termination of the 
Respond- to-Pen function . 

Figure 44 is a flow chart describing the logic of the 
Handle-Pen-In-Title function in the SmartPad calculator. 

15 Block 4402 represents the start of the Handle-Pen-In-Title 
function. Block 4404 saves the current data for any 
potential Undo functions. Block 4406 empties the ink 
written structure, adds the initial pen position to the ink 
written structure, and, while the pen is touching the 

20 screen, reads the pen position, draws a line in the ink 
image from the last point in the ink written structure to 
the pen position, and if the last two points in the ink 
written structure are in line with the pen position, then 
the last point in the ink written structure is set to the 

25 pen position so that every point along a straight line is 
not saved, otherwise the pen position is added to the ink 
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structure to save the pen position for curving, i.e, non 
straight, lines. Block 4408 performs the handwriting 
recognition function to determine what symbol was written 
by the user on the screen. Block 4410 is a decision block 
5 that determines whether the written symbol is an erase or 
delete symbol. If Decision Block 4410 determines that the 
written symbol is an erase or delete symbol, then control 
is transferred to Block 4412. Block 4412 computes the area 
enclosing the written ink on the screen. Block 4414 

10 computes an intersection of the enclosed area and the title 
area. Block 4416 is a decision block that determines 
whether the intersection includes most of the title area. 
If so, then control is transferred to Block 4418. Block 
4418 empties the list of labels in the title and Block 4420 

15 erases the title area of the display image structure. 

Control is then transferred to Block 4222 which terminates 
the Handle-Pen-In function. If Decision Block 4410 
determines that the written symbol is not an erase or 
delete symbol, or if Decision Block 4416 determines that 

20 the intersection does not include most of the title area, 
then control is transferred to Block 4424. Block 4424 adds 
a new label to the list of labels in the title. Block 4426 
sets the new label ' s graphic form in the list to a copy of 
the written ink. Block 4428 draws the new label's graphic 

25 form on the screen. Block 4422 terminates the Handle-Pen- 
In-Title function. 
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Figures 45A and 45B together are a flow chart 
describing the logic of the Handle-Pen-In-Palette function 
in the SmartPad calculator* Block 4502 represents the 
start of the Handle-Pen-In-Palette function. Block 4504 
5 determines the row and column of the palette at the initial 
pen position. Block 4506 blinks the area of the palette at 
the determined row and column. Block 4508 sets the current 
symbol to the palette symbol at the determined row and 
column. Block 4510 counts the number of objects on the 

10 screen that have been selected by the user. Block 4512 is 
a decision block that determines whether any objects are 
selected on the screen. If one or more objects are 
selected, then control is transferred to Block 4514; 
otherwise, control is transferred to Block 4534. Block 

15 4514 is a decision block that determines whether one or a 
plurality of objects have been selected on the screen. If 
one object has been selected, then control is transferred 
to Block 4516. Block 4516 finds the object on the page 
that has been selected by the user. Block 4518 is a 

20 decision block that determines whether the selected object 
has its original "ink". If so, then control is transferred 
to Block 4520; otherwise, control is transferred to Block 
4522. Block 4520 updates the symbols model with the 
selected object's original "ink". Block 4522 saves the 

25 current data in case the user later invokes the Undo 

function. Block 4524 erases the selected object from the 
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screen and removes it from the calculation. Block 4526 is 

a decision block that determines whether the symbol 

represents an action. If so, then control is transferred 

to Block 4528. Block 4528 represents the Do-Action 

5 function for the selected symbol, as described in more 

detail in Figures 49A and 49B. Block 4530 represents the 

termination of the Handle-Pen-In-Palette function. If 

Decision Block 4526 determines that the symbol does not 

represent an action, then control is transferred to Block 

10 4532. Block 4532 performs the Add-Object function, as ^ 

described in more detail in Figures 50A, 50B, «a**4 50C.. If 

/v 

Decision Block 4512 determines that no objects are 
selected, then control is transferred to Block 4534. Block 
4534 is a decision block that determines whether- there is 

15 written ink on the screen. If not, then control is 
transferred to Block 4526; otherwise, control is 
transferred to Block 4536. Block 4536 updates the symbol's 
model with the written ink; Block 4538 performs the Undo 
" function which reverses the last change made and control is 

20 then transferred to Block 4526. 



Handle-Pen- In-Menu function in the SmartPad calculator. 
Block 4602 represents the start of the Handle-Pen-In-Menu 
function. Block 4604 determines the row and column of the 
25 menu at the initial pen position. Block 4606 blinks the 
area of the menu at the determined row and column. Block 




flow chart describing the logic of the 
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4608 is a decision block that determines whether the row 
and column represents the Undo menu item. If so, then 
control is transferred to Block 4610. Block 4610 performs 
the Undo function which reverses the effects of the last 
5 change made. Control is then transferred to Block 4612 
that terminates the Handle-Pen-In-Menu function. If 
Decision Block 4608 determines that the row and column do 
not represent the Undo menu item, then control is 
transferred to Block 4614. Box 4614 is a decision block 

10 that determines whether the row and column represent the 
Train menu item. If so, then control is transferred to 
Block 4616. Block 4616 erases the title area of the 
display image and control is transferred to Block 4618. 
Box 4618 is a decision block that determines whether or not 

15 the palette is displayed. If so, then control is 

transferred to Block 4620. Block 4620 draws a title on the 
screen and Block 4622 sets the variable that indicates that 
the title is displayed. Control is then transferred to 
Block 4612 and the Handle-Pen-In-Menu function is 

20 terminated. If Decision Block 4618 determines that the 

palette is not displayed, then Block 4624 draws the palette 
and Block 4626 sets the status variable indicating that the 
palette is displayed. Control is then transferred to Block 
4612 and the Handle-Pen-In-Menu function is terminated. If 

25 Decision Block 4614 determines that the row and column does 
not represent the Train menu item, then control is 



transferred to Block 4612 which terminates the Handle-Pen- 
In-Menu function. 

Figures 47A and 47B together are a flow chart 
describing the logic of the Handle-Pen-In-Page function in 
. 5 the SmartPad calculator. Block 4702 represents the start 
of the Handle-Pen-In-Page function. Block 4704 is a 
decision block that determines whether the initial pen 
position is in a selection area. If not, then control is 
transferred to Block 4706 which finds a scroller object on 

10 the page displayed at the initial pen position. Otherwise 
control is transferred to Block 4708. Block 4708 is a 
decision block that determines whether the initial pen 
position is near the top left corner of the selection area. 
If so, Block 4710 sets the pen position to the initial 

15 position, and while the pen is touching the screen waits 
for a new pen position so that it can compute a new 
selection area from the top left of the selected object. 
Block 4712 is a decision block that determines whether the 
initial pen position is near the top right corner of the 

20 selection area. If so, Block 4714 sets the pen position to 
the initial position, and while the pen is touching the 
screen waits for a new pen position so that it can compute 
- a new selection area from the top right of the selected 
object. Block 4716 is a decision block that determines 

25 whether the initial pen position is near the bottom left 
corner of the selection area. If so, Block 4718 sets the 
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pen position to the initial position, and while the pen is 
touching the screen waits for a new pen position so that it 
can compute a new selection area from the bottom left. 
Block 4720 is a decision block that determines whether the 
5 initial pen position is near the bottom right corner of the 
selection area. If so, Block 4722 sets the pen position to 
the initial position, and while the pen is touching the 
screen waits for a new pen position so that it can compute 
a new selection area from the bottom right of the selected 

10 object. Block 4724 is a decision block that determines 
whether the initial pen position is within the selection 
area. If so, Block 4726 sets the pen position to the 
initial position, and while the pen is touching the screen 
waits for a new pen position. so that is. can compute a new 

15 selection area. Block 4728 is a decision block that 

determines whether there is a "scroller" object displayed 
on the screen. If so, Block 4730 scrolls the page. 
Otherwise, Block 4732 performs the Handle-Written-Symbol 
function, as described in more detail in Figure 48. 

20 Finally, Block 4734 represents a termination of the Handle- 
Pen-In-Page function. 

Figure 48 is a flow chart describing the logic of the 
Handle-Written-Symbol function in the SmartPad calculator. 
Block 4802 represents the start of the Handle-Written- 

25 Symbol function. Box 4804 empties the ink written 

structure, adds the initial pen position to the ink written 
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structure, and, while the pen is touching the screen, reads 
the pen position, draws a line in the ink image from the 
last point in the ink written structure to the pen 
position, and if the last two points in the ink written 
5 structure are in line with the pen position, then the last 
point in the ink written structure is set to the pen 
position so that every point along a straight line is not 
saved, otherwise the pen position is added to the ink 
structure to save the pen position for curving, i.e, non 

10 straight, lines. Block 4806 performs the handwriting 

recognition function to determine the symbol written by the 
user on the display. Block 4808 is a decision block that 
determines whether the symbol written by the user on the 
screen is a lasso. If so, control is transferred to Block 

15 4810 which resets the selection; otherwise, Block 4812 
saves the data for an Undo function. Block 4814 is a 
decision block that determines whether the symbol written 
by the user on the display was recognizable. If not, then 
control is transferred to Block 4816 which adds a new label 

20 to the page. If the symbol was recognizable, then control 
is transferred to Block 4818. Block 4818 is a decision 
block that determines whether the symbol written by the 
user on the screen represents an action. If so, then Block 
4820 performs the Do-Action function for the symbol, as 

25 described in more detail in Figures 49A and 49B; otherwise, 
control is transferred to Block 4822. Block 4822 performs 
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the Add-Object function, as described in more detail in 

) OrvJl SOlS 

Figures 50A, 50B, - and 50C. Block 4824 represents the 
termination of the Handle-Written-Symbol function. 
Figures 49A and 49B together are a flow chart 
5 describing the logic of the Do-Action function in the 
SmartPad calculator. Block 4904 is decision block that 
determines whether the symbol is a lasso. If so, Block 
4906 computes, for each object on the page, the amount of 
area if any of the object inside the lasso, and then 

10 identifies the object as being selected if most of the 
object's area is inside the lasso. Block 4908 is a 
decision block that determines whether the symbol is a 
"scratch out". If so, Block 4910 sets the scratch out area 
to enclose the written ink and then removes \the objects in 

15 the scratch out area. Block 4912 is a decision block that ; 
determines whether the symbol is a pigtail. If so, Block 
4914 removes the objects under the pigtail and closes up 
the blank space. Block 4916 is a decision block that 
determines whether the symbol is an up or down caret. If 

20 so, Block 4918 inserts a blank space between the objects 
where the caret was written. Block 4020 is a decision 
block that determines whether the symbol is a left or right 
caret. If so, Block 4922 inserts a blank line between the 
lines of characters where the caret was written. Block 

25 4924 is a decision block that determines whether the symbol 
is a horizontal wave. If so, Block 4926 swaps the objects 
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under the horizontal wave. Block 4928 is a decision block 
that determines whether the symbol is a vertical wave. If 
so, Block 4930 swaps the objects under the vertical wave. 
Block 4932 represents the termination of the Do-Action 
5 function. 

Figures 50A, SOB, dttd SOC^together are a flow chart 
describing the logic of the Add-Object function in the 
SmartPad calculator. Block 5002 represents the start of 
the Add-Object function. Block 5004 computes the area 

10 enclosing the written "ink" on the screen. Block 5006 

finds the overwritten object on the screen. Block 5008 is. 
a decision block that determines whether the overwritten 
object is a label. If so, Block 5010 adds a new label to 
the page. Block 5012 is a decision block that determines 

15 whether the overwritten object is a counter and the written 
symbol is not an arrow. If so, Block 5014 applies the 
written symbol to the overwritten counter. Block 5016 is a 
decision block that determines whether the overwritten 
object is an equals character, result line, or arrow. If 

20 so, Block 5018 removes the overwritten object's parent (a 
calculation) from the page. Block 5020 is a decision block 
that determines whether the overwritten object is a 
character or scroller object. If so, Block 5022 removes 
the overwritten object from the page. Block 5024 is a 

25 decision block that determines whether the written symbol 
is a character. If so, Block 5026 adds a new character to 
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the page and Block 5028 sets the new object's value to the 
character corresponding to the written symbol. Block 5030 
is a decision block that determines whether the written 
symbol is an equals character. If so, Block 5032 performs 
5 the Generate-Horizontal-Calculation function with the new 
object, as described in more detail in Figure 52; . 
otherwise, control is transferred to Block 5034. Block 
5034 is a decision block that determines whether the 
written symbol is used in a number, i.e., whether it is a 

10 digit, a period, or a comma. If so, Block 5036 performs 
the Generate-Number function with the new object, as 
described in more detail in Y&iguro 51 . Control is then 
transferred to Block 5038 which terminates the Add-Object 
function. If Decision Block 5024 determines that the 

15 written symbol is not a character, then control is 

transferred to Block 5040. Block 5040 is a decision block 
that determines whether the written symbol is result line. 
If so, Block 5042 adds a new result line to the page and 
Block 5044 performs the Generate-Vertical-Calculation 

20 function for the new object, as described in more detail in 
Figure 53. Control is then transferred to Block 5038 which 
terminates the Add-Object function. If the written symbol 
is not a result line, then control is transferred from 
Decision Block 5040 to Decision Block 5046. Block 5046 is 

25 a decision block that determines whether the written symbol 
is an arrow. If so, Block 5048 adds a new arrow to the 



page, and Block 5050 performs the Generate-Copy-Calculation 
function with the new object, as described in more detail 
in Figures 54A yand 54^ Control is then transferred to 
Block 5038 and the Add-Object function terminates. Block 
5 5052 is a decision block that determines whether the symbol 
is a square. If so, Block 5054 adds a new counter to the 
page. Control is then transferred to Block 5038 which 
terminates the Add-Object function. Block 5056 is a 
decision block that determines whether the written symbol 
10 is a compass. If so, Block 5058 adds a new scroller to the 
page and control is then transferred to Block 5038 which 
terminates the Add-Object function. 

"B^ ^ - Figure 51 is a flow chart describing the logic of the 
Generate-Number function in the SmartPad calculator. Block 

15 5102 represents the start of the Generate-Number function. 
Block 5104 sets the search area to the character's area 
expanded in the left and right direction. Block 5106 sets 
the status flag to indicate that no number has been found. 
Block 5108 initializes the loop to start with the first 

20 object on the page. Block 5110 is a decision block that 
determines whether the current object is a number. If so, 
Block 5112 computes the intersection of the object's area 
and the search area. Block 5114 is a decision block that 
determines whether there is an intersection between the 

25 object's area and the search area. If so, control is 

transferred to Block 5116. Block 5116 is a decision block 
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that determines whether the object is closer to the left 
side of the character's area than the number. If so, Block 
5118 sets the number to the object. Block 5120 is a 
decision block that controls the end of loop by determining 
5 whether or not the last object has been examined. If not, 
Block 5122 sets the next object on the page for examination 
and control is transferred to Block 5110. After the last 
object has been examined, as determined by Decision Block 
5120, then control is transferred to Block 5124. Block 
10 5124 is a decision block that determines whether a number 
has been found in the loop. If not, Block 5126 adds a new 
number to the page. Block 5128 adds the character to the 
number. Block 5130 terminates the Generate-Number 
function. 

15 Figure 52 is a flow chart describing the logic of the 

Generate-Horizontal-Calculation function in the SmartPad 
calculator. Block 5202 represents the start of the 
Generate-Horizontal-Calculation function. Block 5204 adds 
a new horizontal calculation to the page. Block 5206 adds 

20 the character to the new calculation. Block 5208 performs 
the Find-Object-To-Add-To-Horizontal-Calculation function 
with the new calculation, as described in more detail in 
■ FlgiLHd 55 . Block 5210 is a decision block that determines 
whether there is an object to add. If so, Block 5212 adds 

25 the object to the horizontal calculation. Block 5214 
performs the Create-Calculation-Result function, as 
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described in more detail in Figure 57. Block 5216 
terminates the Generate-Horizon Calculation function. 

Figure 53 is a flow chart describing the logic of the 
Generate-Vertical-Calculation function in the SmartPad 
5 calculator. Block 5302 represents the start of the 

Generate-Vertical-Calculation function. Block 5304 adds a 
new vertical calculation to the currently displayed page. 
Block 5306 adds the result line to the new calculation. 
Block 5308 performs the Find-Ob ject-To-Add-To-Vertical- 

10 Calculation function, as described in more detail in 

Figures 56A^and- 56^ Block 5310 is a decision block that 
determines whether there is another object to add to the 
vertical calculation. If so, Block 5312 adds the object to 
the. vertical calculation. If not, Block 5314 performs the 

15 Create-Calculation-Result function for the new calculation, 
as described in more detail in Figure 57. Block 5316 
terminates the Generate-Vertical-Calculation function. 
Figures 54A^wtd 54By^together are a flow chart 
. describing the logic of the Generate-Copy-Calculation 

20 function in the SmartPad calculator. Block 5402 represents 
the start of the Generate-Copy-Calculation function. Block 
5404 sets the source number to "none". Block 5406 
initializes the processing loop, which begins with the 
first object on the page. Block 5408 is a decision block 

25 that determines whether the object is a number or a 

counter. If so, Block. 5410 computes the center of the 



object's area. Block 5412 is a decision block that 
determines whether the center is close enough to the first 
point of written ink. If so, control is transferred to 
Block 5414. Block 5414 is a decision block that determines 
5 whether the center of the object's area is closer than the 
source number's center. If so, Block 5416 sets the source 
number to this object and control is transferred to Block 
5418. If the center of the object's area is not close 
enough to the first point of written ink, or if the center 

10 of the object's area is not closer than the source number's 
center, then control is transferred to Block 5418. Block 
5418 is a decision block that determines whether the loop 
has processed all objects on the page. If not, the next 
object is selected by Block 5420 and control is transferred 

15 to decision Block 5408 for additional processing. If the 
last object has been processed, then control is transferred 
to Block 5422. Block 5422 is a decision block that 
determines whether there is a source number. If yes, Block 
5424 adds a new copy calculation to the page. Block 5426 

20 adds the source number to the calculation. Block 5428 adds 
the calculation to the source number's list of copiers. 
Block 5430 adds the arrow to the calculation. Block 5432 
performs the Create-Calculation-Result function for the 
calculation at the last point of written ink, as described 

25 in more detail in Figure 57. Block 5434 routes the arrow 
from the source number's area to the result's area. Block 
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5436 terminates the Generate-Copy-Calculations function. 
If Decision Block 5422 determines that there is not a 
source number, then Block 5438 removes the arrow from the 
page and Block 5436 terminates the Generate-Copy- 
5 Calculation function. 

^-^■ Figure 55 is a flow chart describing the logic of the 
Find-Object-To-Add-To-Horizontal-Calculation function in 
the SmartPad calculator. Block 5502 represents the stairt 
of the Find-Object-To-Add-To-Horizontal-Calculation 

10 function. Block 5504 sets the search area to the area to 
the left of the calculation. Block 5506 sets the object to 
"none". Block 5508 initializes the processing loop by 
selecting the first object on the page. Block 5510 is a 
decision block that determines whether the current object 

15 is a character , number, or counter. If so, control is 

transferred to Block 5512. Block 5512 is a decision block 
that determines whether the object has a parent. If not, 
Block 5514 computes an intersection between the object's 
area and the search area. Block 5516 is a decision block 

20 that determines whether there is an intersection between 

the object's area and the search area. If so, then control 
is transferred to Block 5518. Block 5518 is a decision 
block that determines . whether there is an object to add to 
the calculation. If so, then control is transferred to 

25 Block 5520. Block 5520 is a decision block that determines 
whether the current object is to the right of the object to 
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add. If so, or if Decision Block 5518 determines that 
there is no object to add, Block 5522 identifies the object 
as the one to add. Block 5524 is a decision block that 
terminates the loop. If there are more objects on the 
5 page, then control is transferred to Block 5526 which 

selects the next object to process. Otherwise, control is 
transferred to Block 5528 which terminates the Find-Object- 
To-Add-To-Horizontal-Calculation function. 

Figures 56A^<att& 56Byytogether are a flow chart 

10 describing the logic of the Find-Object-To-Add-To-Vertical- 
Calculation function in the SmartPad calculator. Block 
5602 represents the start of the Find-Ob ject-To-Add-To- 
Vertical-Calculatlon function. Block 5604 sets the primary 
search area to the area to the left of the first child of 

15 the calculation. Block 5606 sets the secondary search area 
to the area above the calculation. Block 5608 sets the 
object to add to "none". Block 5610 initiates the 
processing loop to start with the first object on the page. 
Block 5612 is a decision block that determines whether the 

20 current object is a character, number or counter. If so, 
then control is transferred to Block 5614. Block 5614 is a 
decision block that determines whether the current object 
has a parent. If not, Block 5616 computes the intersection 
of the object's area and the primary search area. Block 

25 5618 is a decision block that determines whether there is 
an intersection between the object's area and the primary 
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search area. If not, Block 5620 computes the intersection 
of the object's area and the secondary search area. Block 
5622 is a decision block that determines whether there is 
an intersection between the object's area and the secondary 
5 search area. If there is an intersection between the 
object's area and either the primary search area or the 
secondary search area, then control is transferred to Block 
5624. Block 5624 is a decision block that determines 
whether there is an object to add to the calculation. If 

10 so, then control is transferred to Block 5626. Block ,5626 
is a decision block that determines whether the current 
object is below the object to add. If not, then control is 
transferred to Block 5628. Block 5628 is a decision block 
that determines whether the current object is to the right 

15 of the object to add and not above it. If the object is to 
the right of the object to add and not above it, or if the., 
object is below the object to add, or if there is no object 
to add,, then control is transferred to Block 5630 which 
identifies the current object as the object to add. 

20 Control is then transferred to Block 5632. Block 5632 is a 
decision block that determines whether every object on the 
page has been processed. If so, Block 5634 terminates the 
Find-Object-To-Add-To-Vertical-Calculation function. If 
not, control is transferred to Block 5636 which selects the 

25 next object on the page for processing. 

Figure 57 is a flow chart describing the logic of the 
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Create-Calculation-Result function in the SmartPad 
calculator. Block 5702 represents the start- of the Create- 
Calculation-Result function. Block 5704 adds a new 
character to the page at the desired area. Block 5706 sets 
5 the new character's value to blank. Block 5708 adds a new 
number (the calculated result) to the page at the desired 
area. Block 5710 adds the new character to the new number. 
Block 5712 adds the new number to the calculation. Block 
5714 sets the new number's calculator to the calculation. 
10 Block 5716 represents the termination of Create- 
Calculation-Result function. 



Calculate function in the SmartPad calculator. Block 5802 
represents the start of the Calculate function. Block 5804 

15 is a decision block that determines whether the number has 
a calculator. If so, then control is transferred to Block 
5806. Block 5806 is a decision block that determines 
whether the number's calculator is horizontal. If so, then 
Block 5808 computes the result by evaluating the horizontal 

20 formula. If not, then control is transferred to Block 
5810. Block 5810 is a decision block that determines 
whether the number's calculator is vertical. If so, then 
Block 5812 computes the result by evaluating the vertical 
formula. If the number's calculator is neither vertical 

25 nor horizontal, then Block 5814 computes the result using 
the value of the first child of the number's calculator. 




flow chart describing the logic of the 
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After the result has been computed, Block 5816 forces the. 
number's value to the result. Block 5818 sets the animated 
calculation variable to identify the number's calculator. 
Block 5820 sets the animated object variable to identify 
5 the first child of the calculation. Block 5822 turns the 
animation flag on. Block 5824 sets the animate time to the 
current time. If the number does not have a calculator, or 
after the animate time has been set to the current time, 
Block 5826 sets the text to the sequence of values of the 
10 number's children, i.e., the characters. Block 5828 sets 
the number's value to a decoding of the text as a number. 
Block 5830 turns the number's computed flag on. Block 5832 
represents the termination of the Calculate function. 

15 23. CONCLUSION 

This concludes the description of the preferred 
embodiment of the invention. In summary, the present 
invention discloses a pen-based calculator which recognizes 
handwritten input. The calculator consists of a display 

20 simulating a sheet of paper, and a stylus simulating a pen. 
The user writes a calculation on the calculator as if it 
were a piece of scratch paper. The calculator uses 
handwriting recognition to identify the various elements of 
the calculation, performs the calculation, and then 

25 displays the result at an appropriate lpcation. 

The foregoing description of the preferred embodiment 
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of the invention has been presented for the purposes of 
illustration and description. It is not intended to be 
exhaustive or to limit the invention to the precise form 
disclosed. Many modifications and variations are possible 
5 in light of the above teaching. It is intended that the 
scope of the invention be limited not by this detailed 
description, but rather by the claims appended hereto. 
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